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ABSTRACT 

The theme of the 32nd International Association for 
the Development of Computer-Based Instructional systems (ADCIS) 
Conference was "Restructuring Training and Education through 
Technology." This collection of conference presentations contains 52 
papers ^•id 183 abstracts for which there are no formal papers. The 
papers and abstracts are presented in two separate sections, but both 
are categorized by special interest group: (1) Academic Computing 
(SIGAC, 1 paper, 11 abstracts); (2) Computer-Based Training (CBT, 3 
papers, 27 abstracts); (3) Elementary, Secondary, Junior College 
Educators (ELSECJC, 3 papers, 17 abstracts); (4) Emerging 
Technologies (SIGET, 5 papers, 10 abstracts); (5) Educators of the 
Handicapped (SIGHAN, 2 abstracts); (6) F<*alth Education (SIGHEALTH, 2 
papers, 9 abstracts); (7) Home Economics Consortium (HOMEC, 3 papers, 
5 abstracts); (8) Hypermedia (SIGHYPER, 5 papers, 18 abstracts); (9) 
Interactive Video-Audio (SIGIVA, 7 papers, 19 abstracts); (10) 
Management Issues (SIGMI, 1 paper, 9 abstracts); (ll) Music Education 
(SIGMUSIC, 1 paper, 4 abstracts); (12) PILOT Users Group (PILOT, 1 
paper, 3 abstracts); (13) PLATO Users' Group (PUG, 4 abstracts); (14) 
Theory and Research (SIGTAR, 15 papers, 38 abstracts); and (15) 
Telecommunications (SIGTELE, 5 papers, 7 abstracts). Many of the 
papers include abstracts and or references. (DE) 
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COMPUTER INVENTORIES ARE VALUABLE BENCHMARKS 



Elisabeth A/. Craig, The University of Tennessee at Chattanooga 

A benchmark, called an Inventory of Courses Utilizing Computers in the Curriculum, has been published on an 
annual basis since 1976. This Inventory catalogs important shifts in computer usage, reports on departmental 
utilization, and summarizes information from current and previous years for comparison. The presentation describes 
the Inventory for those interested in replicating it. 

Since 1976, our Academic Computing Services has published annually a benchmaric called the Inventory of Courses 
Using Computers in the Curriculum. Academic departments are surveyed and provide data on courses which use 
computers, the number of students, hours of usage per week, the computer system (mini or micro) and type of usage 
in the course, i.e., statistical packages, simulations, problem solving, etc. 

The inventory lists each course information, sorted according to department. There are eleven summary tables which 
show the results from other years plus the current one for comparison purposes. The summaries also show computer 
usage by department, system and microcomputer laboratory. 

These inventories are valuable benchmarics for documenting the growth of computer usage within courses and clearly 
indicate the switch from mini to micro usage over the past ten years. Outside consultants have commended ui on the 
information and its concise format. Also, the inventory was an important source document for a recent strategic 
planning task force on academic computing. 

Over the years, we have discovered how to make data collection for the inventory as effortless as possible. The 
presentation discusses the format used to collect and analyze the data and presents examples of the departmental and 
summary tables. This is a "how to" presentation for those interested in developing their own inventory. 

Elisabeth M. Craig 

Center for Excellence for Computer Applications 
The University of Tennessee at Chattanooga 
615 McCallie Avenue 
Chattanooga, TN 37^03 
615-755-4389 



INSTRUCTIONAL COMPUTING LABORATORY PROJECT 



David W. Dalton, Kent State University 

The Instructional Computing Laboratory Program (ICL) has as its principal mission, the design, development, and 
implementation of large-scale, integrated computer-based instructional systems, specifically, computer-based 
alternatives to conventional course delivery systems. The ICL houses five major concurrent research and development 
projects. The purpose of this session is to detail the goals and activities involved in each of the ICL projects to date. 

Kathleen M. Hannafin 
406 MCH B-150 CET 
Florida State University 
Tallahassee, FL 32306 
(904) 644-4720 
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TECHNOLOGICAL SOLUTIONS: 
FACULTY + INSTRUCTIONAL DESIGNER = HIGH-QUALITY PRODUCT 
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Carol A. Dwyer, Pennsylvania State University 



While academic computing professionals debate whether the best software for higher education is developed by a 
team or alone by a faculty member, we may be ignoring the fact >hat many faculty will not get involved in software 
use and development without some form of support. Acknow iging cost, time factors, and the difficulty of the 
content expert in conveying the content and pedagogical expectations to the team, more faculty will become users of 
instructional technology if they have assistance in locating, evaluating, and integrating instructional software. If a 
goal within higher education institutions is to encourage faculty to use interactive instructional technology, 
advantages and disadvantages of a support team should be considered 

The Center for Academic Computing (CAC) of Penn Slate provides broad support services to faculty, staff, and 
students. Within the CAC, CBEL — ^The Teaching and Learning Technologies Group has as its mission to support 
and empower faculty in the use of interactive technologies in teaching and learning. In cooperation with faculty, 
CBEL has developed instructional applications in many disciplines, including Mineral Science (Crystallography), 
Education (Teaching of Reading), PoUtical Science, Agronomy (Weed Identification), and Foreign Languages. Teams 
consisting of content experts, instructional designers, programmers, and graphic artists combine their expertise to 
develop high-quality products. 

Benefits of team-ba<^ design include development by more faculty, product quality, clarity, consistency, and 
completion of useable products. Some of the inefficiencies of team-based development include cost, slow start up 
time, and difficulty of project management The presentation will include demonstration of several instructional 
applications, while highlighting problems and solutions unique to each project. 



While developing an academic computing plan for the University of Northern Colorado over the last two years, 
much effort has been expended and many lessons have been learned. The process of determining the problems and 
needs, establishing the recommendations, completing the plan, and utilizing computer tools will be chronicled. 
Practical suggestions from the experience will be highlighted. 



Carol A. Dwyer 
Penn State University 
12 WiUaid Building 
University Park, PA 16802 
(814)863-1654 
Bimet* eca@psuvm 



LESSONS LEARNED WHILE DEVELOPING 
A UNIVERSITY ACADEMIC COMPUTING PLAN 



Peter C. Issacson, University of Northern Colorado 



Peter C. Issacson 

Ross Hall, Math and Statistics Department 
University of Northern Colorado 
Greeley, CO 80639 
(303) 351-2215 
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HYPERCARD IN HIGHER EDUCATION: 
SUCCESSES, CHALLENGES, AND STRATEGIES FOR USE 



Annette Lamb, University of Toledo 

HypeiCard has become a popular tool for educators. This presentation describes the successes, challenges and 
strategies for incorporating effective uses of hypermedia technologies into higher education. It begins with a 
discussion of new technologies and reluctant technologists. It then focuses on current and projected uses of these 
technologies in higher education and describes obstacles in terms of personnel and funding. HypeiCard applications 
in higher education are then discussed. Specifically, the use of HypeiCard for the development and delivery of desktop 
presentations, tutorials, and information exploration-based instructional materials is discussed. Successes and 
challenges facing incorporation of hypermedia are described within the context of The University of Toledo. Sneaky 
and not-so-saeaky strategies for change ve then highlighted, followed by a vision of the future of hypermedia in 
higher education. 

Annette Lamb 

The University of Toledo 

2801 West Bancroft, 2021 GH 

Toledo, OH 43606 

(419) 537-4713 



INSTRUCTIONAL DESIGN FOR 
COMPUTER-MEDIATED COMMUNICATION 



Maurice Mitchell, University of Nevada System Computing Services 

Compufer-mediaied communication (CMC) seems to be the latest fad in the use of computers in instruction. Recent 
activities in public education have highlighted the technology. U. S. Office of Technology Assessment reports 
include sections on CMC. The prqponents of the National Research and Education Network arc actively advocating 
the use of the NREN for CMC in instruction. FrEdMail and KIDSNET have fired up teachers in the K-12 schools 
about the value of CMC in ihcii curricula. Faculty in higher education have engaged in computer conferences to 
investigate the value of CMC for instruction in higher education. 

Much of the early use of CMC was ad hoc and haphazard in design. However, as CMC has grown in popularity 
faculty who embrace it are beginning to ask questions about how to use CMC to improve the instructional 
experience. Faculty who wish to try CMC for the first time are looking for guidance. While some guides have been 
produced to assist K-12 teachers, little has been gathered for faculty in higher education. This session will review a 
simple design guide for using CMC in instructional situations in higher education. 

Maurice Mitchell 
University of Nevada System 
Computing Services 
4505 Maryland Pkwy. 
Las Vegas, NV 89154 
(702)739-3557 
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AUTHORING TEMPLATES IN PILOT: 
ASSISTING MORE FACULTY WITH FEWER RESOURCES 



Barbara A. Polka and Marilynne W. Stout, Pennsylvania State University 

Faculty support units for instructional ^ftware development have limited full time staff to support an exploding 
interest among faculty members Penn State's Computer Based Education Lab (CBEL) uses some unique 24)proaches 
to solve this problem-Authoring Templates is one. Demonstrations of faculty courseware developed using PILOT 
templates and discussion of the success and pitfalls will complete this presentation, 

Barbara A« Polka arul Marilynne W. Stout 
Penn State University 
12 Willard Building 
University Park, PA 16802 
(814)863-1654 



AN UPDATE ON THE MODEL CLASSROOM PROJECT AT PENN STATE 



Lawrence C. Ragan, Pennsylvania State University 

The **Model Classroom** project was designed to create an exemplary model of how computer technology could be 
integrated into the classroom. This project is a cooperative effort between several units at Penn State, including the 
Center for Academic Computing, the College of Agriculture, and the Computer Based Education Laboratory-The 
Teaching and Learning Technologies Group with support from Apple Computer Inc. The course selected to 
demonstrate this state-of-the-art use of technology is Animal Science 1, an introductory course for agricuhure 
majors. 

The project consists of three major phases: 

• Phase I: Instructional Matoials Development and Delivery 

• Phase n: Student Support Materials 

• Phase III: Lab Exercises and Simulations 

This presentation will provide an update on the ''Model Classroom" project initiated in 1989 at Penn State. Aspects 
of implementation and an evaluation of Phase I will be discussed including modiflcations and adjustments. 
Coursev/are presentation software developed for the project will also be demonstrated. Progress and plans for Phase II 
and III will be introduced. Discussion among participants will be encouraged focusing on problems and solutions 
encountered in similar experiences. 

Lawrence C.Ragan 
Insuuctional Designer 
Penn State University 
12 Willard Bldg. 
University Park, PA. 16802 
(814)863-1654 
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100% OF EMPIRE STATE COLLEGE FACULTY ARE INVOLVED IN COMPUTING: 

THIS IS HOW WE DID IT 



Carol A. Twigg, Empire State College 

Many colleges and universities are interested in integrating computing and other technologies into the curriculum, 
but frequently they encounter resistance or disinterest on the part of faculty. No matter how dazzling the technology, 
no matter how promising the pedagogical potential, no matter how interested the students-when it comes to 
academic computing, if you can't gel the faculty to use the technology, the rest is just talk. Developing successful 
strategies to engage college faculty in using computing is the key factor in achieving a flourishing academic 
computing program. 

SUNY/Empire Stale College has engaged 100% of its full-time faculty in computing. This presentation will share 
strategics ihat have worked at ESC, including 1) make getting computers into faculty hands the number one a 
priority; 2) create critical mass of users at each stage of development; 3) don't worry about uneven development- 
encourage it; 4) don't worry too much about the particulars of computer use-just be glad they're being used; 5) create 
and support applications that people can use easily and that meet real needs; 6) don't be afraid to try stuff even if it 
fails; 7) slajidardize hardware as early ii^^ possible and software when people are ready for it; 8) provide as much 
training and support as you can manage, but realize that it's never enough; 9) engage in a constant internal public 
relations campaign; and 10) try to ignore the complaining. 

Carol A. Twigg 
Empire State College 
1 Unkxi Ave 

Sarasota Springs NY 12866 
(518)587-2100 



THE MARYLAND NUCLEAR SCIENCE BACCALAUREATE DEGREE PROGRAM: 

ON-SITE, ON-LINE, ON-TIME 



Marcia A. Whitney, Thomas A. Janke, University of Maryland: and 
y. Robert ^fueller, Wisconsin Public Service Corporation 

The country's largest nuclear science baccalaureate degree program is delivered through computer by the University of 
Maryland to over 700 students at 8 nuclear utilities in 12 sites. The panel will discuss and evaluate the program from 
three perspectives: 1) developing multimedia courses for the adult, 2) administering a multimedia degree program, 
and 3) incorporating education on-site in a nuclear utility. 

Marcia A. Whitney 

University of Maryland Univenity College 
University Blvd. @ Adelphi Rd. 
CoLege Parte MD 20742-1616 
(301)985-7883 
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INSTRUCTIONAL COMPUTING STRATEGIES: 
CREATIVITY ON CAMPUS 



Laura Yordy, University of North Carolina at Greensboro 

Civcn a low budget, how can wc promote, coordinate, and assist the development and/or implementation of creative 
instructional computing in a university setting? Several strategies will be discussed, including: ^^secding** the campus 
with low-risk, high-visibility projects, establishing grants for beginning faculty projects, organizing users* groups, 
and sponsoring project exhiUtions. 

Laura Yofdy 

University of North Carolina, Greensboro 
Academic Computing Center 
Greensboro NO 27412-5001 
(919)334-3117 
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METHODOLOGIES FOR DESIGN AND PRODUCTION OF CBI 



CBI is recognized to have a great potential. But often the potential is not completely realized. There are two major 
reasons for this. One is the fact that CBI implies a learning scenario which is different firom the one involved with 
classical media. Therefore, the design of the interaction between the learner and the technology should be thought of 
on the basis of strategies brought to bear by the learner. 

The second reason is bound to the production of CBI itself. The production process of CBI entails a coherence 
between three facets. The instructional design, the computer implementation and the maiiceting :jialysis. Very often, 
one or sometimes two of these facets are underestimated or not worked out with in depth analysis. If any of the three 
components are partially left out, no real coherence can be obtained. For instance, the product may be well designed 
and adequately implemented, but the analysis of the endniser's profile may be meager. 

The objective of the Project Start Up of the European Community is to make an in-depth analysis of the design and 
production process of CBI in order to build a methodology within which total cdierence between the three aspects 
can be obtained. This analysis relies in particular on strong considerations of the ^)ecificity of the learning scenario 
involved in CBI. 

Andpg Boder Ftoject Stan-Up 

P.O. Box 172 

121 1 Geneva 3 Switzerland 

4122-781-1530 



ERGONOMIC CBT 



KayBonham, Goal Systems Intemadonal 

Eigonomics is a design philosophy that considers various aspects of usability. For example, efficiency, ease of use, 
and comfort level are central ergonomic p»inciples. This session describes ergonomics and explains how sound 
ergonomic design principles can be applied specifically to the design and development of quality computer-based 
training. 

This session will analyze the usability of everyday objects to illustrate the importance of ergonomic principles in the 
design process. It will describe a systematic ergonomic design procedure and will compare this procedure to the CBT 
development process. 

This session will also discuss the benefits of using ergonomic design principles in CBT development and the 
problems that arise from not following ergonomic principles. It will encourage participants to apply an ergonomic 
approach to various phases of the CBT design process, from how to accommodate diverse learning styles to how best 
to present inf(vmation on the screen. 

Kay Bonham 

Goal Systems International 
7965 N. High Street 
Columbus, OH 43235 
(614) 888-1775 
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AN EXAMPLE OF A COOPERATIVE EFFORT 
BETWEEN INDUSTRY AND ACADEMIA 

I 

Jerry L. Brown, Hewlett-Packard Company: and Endle Attala. California Polytechnic State University 

This presentation describes a cooperative activity between business and academia in which courseware for teaching 
transmission line fundamentals was created. The interaction pennitted educational computing students at Cal Poly to 
vfotk on a "real world" training jrogram while giving HP technical experts an ojqKHtunity to learn how to design 
technical training fcx' delivery using the computer. Both institutions benefitted from the experience and are loddng 
forward to continued coopmaon. The presentation will describe the coUabcnative process used, and it will discuss 
the lessons we have learned from our joint activity. These should be useful to others looldng to establish cooperative 
anangements between industry and hi^ier education. 

Jerry L. Brown 
Hewlett-Packard Company 
3200 Hillview Avenue 
Rilo Alto CA 94304 
(415)857-6120 



SOMETIMES THE MOST COST EFFECTIVE TRAINING IS...NO TRAINING 



Charles C, Buchancx United Airlines 

Most of us have training as a part of our job or dqiartment title. If not we believe that training is a big part of our 
contribution. This percqHion may limit our proposed solutions to human performance deficiencies. 

When we change our perceptions from training to improving performance suddenly more options are available. One 
of those cqHions is a Performance Support System. People can be more productive, sooner when provided with a 
Performance Support System instead of training. This presentation reports on the United Airiines/Apple Computer 
Joint development project to prototype such a system for United's Maintenance Operations Division. 

There are several critical elements to performance support 
Design of materials 
Effective user navigation 
Workplace accessibility 

Technology is releasing us from the constraints of traditional learning concepts. We can accommodate on demand, 
learner controlled materials if we try. There is a natural learning model, the way people leam when they aren't even 
trying, that can be applied to learning materials design. 

We can accommodate learners by providing easy to use methods allowing them to organize their experiences the way 
they see nt We can select cost effective delivery that allows access at the job site, wherever it may be. 

Charies C. Buchanan 
United Airlines 

SFOMT, International Airport 
San Francisco CA 94128 
(415)876-5805 
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COMPUTER-BASED INSTRUCTION FOR TEACHING ENGLISH AS A SECOND 

LANGUAGE IN TAIWAN 



Lih'Juan Chanlin and Thomas C Reeves, University of Georgia 

This study presents the status of CBI and IVD for teaching English as a second language (ESL) in Taiwan, and 
provides recommendations for further develqMnent The instructional capabilities of interactive iq)pn)aches are 
contrasted with the rudimentary instruction of existing ESL {mgrams. The power of interactive matcdals to reflect 
traditional Chinese values (family, respect, obedience, loyalty, morality, and industriousness) is stressed 

Lih-Juan Chan Lin 
The University of Georgia 
607 AderhoM Hall 
Athens OA 30602 
(404)542-3810 



COMBINING ACCELERATED LEARNING 
AND COMPUTER-BASED LEARNING 



Frank Clement, Boulder Center for Accelerated Learning: and Chuck Buchannan, United Airlines 

Accelerated Learning (AL) is rapidly making an impact on corporate training and, at a slower pace, on primary 
education. Initial rqxKts indicate an increase of learning of 35-50% and a reduction of learning time by 40-66%. The 
combination of AL and computer-based learning (CBL) holds significant promise of even greater gains. This 
presentation will discuss how AL and CBL can be combined 

Frank Qenoent 

Boulder Center for Accelerated Learning 
525 Ithaca Drive 
BouUer CO 80303 
(303)499-2521 
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IMPLEMENTING THE ADVANCED QUALIFICATION PROGRAM (AGP) 
FOKKER-100 (F-100) PILOT TRAINING AT AMERICAN AIRLINES 



Angela G. Coburn and Janet A. Wise, American Airlines 

Pilot training is a priority facet of the airline industry. Recently, the Federal Aviation Administration (FAA) 
approved the development of proficiency-based pilot training curriculums rather than fixed-length pilot training 
curriculums under the authority of SFAR 58 known as Advanced Qualification Program. Cunent pilot training 
methods differ substantially from the proposed AQP which leans heavily toward ISD methodology. American 
Airlines intends to develop the F-lOO pilot training as an AQP curriculum. 

The AQP curriculum will include computer-based training (CBT), simulations, and part-task L-ainers, as well as 
instructor-based training. The curriculum is developed by first performing a thorough analysis of each task the trainee 
must master in order to become proficient in operation of the airplane. This is a monumental undertaking by any 
standard. 

The linkage that will be established in the training based upon ISD principles lends itself greatly to the use of a 
comprehensive automated system. American Airlines is developing such a system based on relational database 
configurations that will facUitate the analysis, design, and development phases of the AQP pilot training curriculum. 
We will present a description of the methodology to be utilized in implementing the AQP with an automated system 
and the results of the implementation process that have been completed by October 1990. 

Angela G. Coburn 

Analyst, Instructional Systems 

American Airiines, Inc. 

Right Academy MD 871 

4601 Hwy. 360 

Fort Worth, TX 76155 



EVALUATION OF AN {NSTRUCTIONAL DESIGN WORKSTATION 

AT MULTIPLE SITES 



Kent L. Gustqfson and Thomas C. Reeves. The University of Georgia; and Alberta Wray, Apple Computer 

ID Library (IDeaL) (formeriy called IDioM) is a software package for the Apple Macintosh environment that provides 
a comprehensive information base regarding 3 systematic instructional design (ID) model plus an extensive set of 
computer-based tools to support the ID process (Gustafson & Reeves(1990) IDeaL was originally designed for in- 
house use at the Training Support department of Apple Computer, Inc. This presentation reports the findings of a 
comprehensive beta test and evaluation of IDeaL at six external sites. 

The evaluation addressed the following seven questions: 1) Docs IDeaL work as designed?; 2) Is IDeaL a valid 
support system for instructional design in training contexts outside Apple?; 3) How long does it take to learn to use 
IDeaL?; 4) How easy is IDeaL to use?; 5) What is the effectiveness of the tools in IDeaL?; 6) How is ID Library 
used?; 7) What is the projected value and market for IDeaL? Evaluation instrumentation included logs, 
questionnaires, anecdotal record forms and telephone interviews. 

Kent L. Gustafson 
The University of Georgia 
607 Adeihold Hall 
Athens OA 30602 
(404)542-3810 
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CAN COMPUTER-BASED TRAINING BE FUN? ARCO THINKS SO 



Connie Haskell, Digital Equipment Corporation 

ARCO refinery needed to train maintenance woricers. engineers, and contractors to view, print, and markup on-line 
'*blueprints\ Digital Equipment Corporation created a unique CBI software emulation that motivates and holds your 
attention! Special features of the software include a playful storyline with cast of characters, audio instruction, MS* 
Windows with mouse int^ace, highly interactive game-like exercises, cdor graphics, and a unique feedback style. 

Connie Haskell 

Digital Equipment Gxporation 
Custom Training Services 
30 North Avcnoc 
Buriinglon MA 01803 
(617)273-€350 



BUILDING A CONSORTIUM TO PRODUCE 
A COMPUTER*BASED LEARNING MODULE: 
CONCEPTS VERSUS EXPERIENCE 



Brian E, Hecbnan, COMET 

The University Corporation for Atmospheric Research (UCAR) formed COMET to serve as a bridge between 
operations, research, and education in the field of the atmospheric science discq>line. The Distance Learning Program, 
one of three COMET programs, was formed to provide on site training and education for the nation's operational 
forecasters and selected university students. One task for the consortium is to produce computer-based lcanung(CBL) 
modules. Rather than create a large COMET core staff for this function, it was decided to rely on a series of 
cooperative and contractual agreements between universities, government laboratories, and commercial companies to 
produce the modules. Instn v»ional designws and student interns (in addition to one core staflf designer) are employed 
under an agreement between the University of Colorado at Denver and CX)MET; subject matter experts come ftom 
the spons(Ming agencies, universities, or government research facilities. Production facilities are provided by the 
United States Air Force Academy through a quid pro quo with the National Weather Swirice (one member of the 
consortium) while programming is provided by a commercial contract with Poseidon Systems, Inc. of Boulder. 

We have found from experience in developing the first two CBI modules distinct advantages and disadvantages of die 
initial concept This presentation will discuss areas where changes are needed and die parts of the process Uiat have 
been highly successful. 

Brian E. Heckman 
UCAR/COMET 
P.O. Box 3000 
Boukier CO 80302 
(303)497-8498 
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ISD & SDLC: SIMILAR TUNES, DIFFERENT LYRICS 



Michael J. Hillelsohn, McDonnell Douglas Space Systems 

For those involved in developing courseware in a software systems development environment there are many issue's 
related to both training development process and the timeliness of training products. The stages of the Systems 
Development Life Cycle (SDLC) model, used in software development, are similar to Instructional Systems 
Development (ISD) phases. SDLCs lend to be versions of DoD STD 2167A, which are customized for the program 
under development. When instructional materials have to be developed as part of the final system, it is important 
that the instructional designers be sure that instructional development concerns are accommodated by the SDLC 
which is adopted. Typically, the training deliverables which are defined in the SDLC are due during SDLC stages 
which do not coincide with appropriate times during instructional systems development. Modifying the training 
processes and deUverables area of the SDLC can produce training which better meets user requirements, because the 
training products are developed at the most efficient time in the development cycle of the software system. CBT can 
be thought of as a software application which should be developed, according to both ISD and SDLC methods. 
SDLC methodologies such as reviews, walkthroughs, inspections and systematic testing are entirely compatible 
with ISD methodologies and phases. 

Michael J. Hillelsohn 

McDonnell Douglas Space Systems 

11220 Handlebar Road 

Reston VA 22091-3908 

(703)920-3131 



THREATBUSTERS" COMPUTER-BASED INSTRUCTIONAL GAME 



John W. Keller and John A. deVries, Los Alamos National Laboratory 

Using principles found in such computer games as "Where in the World is Carmen San Diego?" and "Duck Hunt," 
the Cognitive Systems Engineering Group of the Los Alamos National Laboratory developed "Threatbusiers," a tool 
for teaching concepts in computer security. The game uses simulated interviews (dialogue) to present information 
that the student must then use to successfully complete the game. 

A combination of text, graphics, and audio was used to provide a multilevel interface. This level of multimedia is 
handled by the use of Modula-2 based projector scripts. A database of knowledge on which the game is founded was 
developed in HyperCard. It was then electronically transferred to the PC and accessed as an integral part of the game. 
The shell of the game is independent of the content of the database; thus, the game can be used with other 
appropriate databases. This project also represents a unique development environment. Many people like HyperCard 
for fast prototyping but not for delivery. We used it to flesh^out ideas from brainstorming sessions. Once the basic 
structiu"eof the game had been worked out, the PC-based delivery version was more easily realized. User testing was 
instrtimental throughout the development process. Cross sections of the user population were asked to play the game 
at several stages during the development period. Their feedback was then used in subsequent versions. These 
techniques provided a unique and efficient way to develop computer-based trainers. 

John W. Keller 

Los Alamos National Laboratory 
P.O. Box 1663 MS M9 
Los Alamos NM 87544 
(505) 6654231 
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CONDUCTING AND USING A TASK ANALYSIS TO FOCUS 
COMPUTER-BASED TRAINING MATERIALS 



Frederick G. Knirk, University of Southern California; and 
Daniel Christinez, Navy Personnel Research and Development Center 

Task analysis helps the CBT planner by identifying the tasks that must be perfonned by the trainee. The task 
listing/hierarchy provides the structure for CBT lessons and specifies the type of test items. In this session, we will: 
1) review developing procedures for using a computer database to manipulate task statements; 2) discuss how to 
perform a task analysis for CBT course materials using a graphical interface; and 3) how to effectively use a task 
analysis in the design of those materials. 

Fredenck G. Knirk 

University of Southern California 

948 West 37th Street 

San Pedro CA 90731 

(213)743-6263 



JUST IN TIME TRAINING 



Judith Morel, Marc Hopkins and Melissa Berkowitz, Tandem Computers 

This presentation will address a technology-based training delivery program developed by the Software Education 
group ai Tandem Computers, Inc. Tandem*s Learning Center Program provides customer and employee training 
when it is needed, where it is needed, and on the subjects needed. The presentation will describe the Learning Center 
program, and demonstrate one of the CBT courses developed specifically for the Learning Center environment 

Judith Morel 

Tandem Computers 

2860 San Tomas Expwy LOC 137-01 

Santa Clara CA 95051 

(408)562-8827 
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INFORMATION REQUIREMENTS FOR LEARNER-CONTROLLED CAI 



James R. Okey and Marshall G. Jones, The University of Georgia 

Computer courseware is available that provides learners with numerous choices about sequence of content, amount 
and lime of i^actice, kind of feedback received, or when to exit instruction and be tested. Increased control by learners 
over such decisions is thought to aid motivation and possibly be important in how knowledge is constructed and 
organized 

Despite the promise of leamer-<:ontrolled instruction, some computer environments (e.g., hypertext materials) have 
the potential to lose learners in a morass of details, layers, and decisions. What kind of information do learners need 
if they are to make sensible and effective control decisions during CAI lessons? 

The purpose of this presentation is to consider different kinds of actions learners may make during computer lessons 
(e.g., navigation, problem selection, content choice, or type of feedback to receive) and examine the kind of 
information learners need in order to exercise control in an intelligent way. Techniques will be described that are 
useful in providing information so that learners can exercise effective control of decisions during CAI - these include 
content miq)s for navigation and course structure, graphs to depict achievement of outcomes, clocks to portray time 
spent, and text to convey the degree of mastery. Hypermedia programs will be used to illustrate how this information 
can be communicated to learners to promote intelligent and eflfective learner control in CAI. 

James R. Okey 

The University of Georgia 

607AdaholdHaU 

Athens G A 30602 

(404)542-3558 



WRITING CBT FOR PROFIT • THE PRACTITIONERS' METHOD 



Jeff Oliver, Castle Learning Systems Ltd 

The conventional ISD model for courseware is fine in theory, but has shortcomings when writing CBT for 
commercial clients. Few CBT jHojects allow auth(^ to take a 'top down' approach and traditional design scripts are 
unsatisfactory for describing modem CBT. Implementation issues need to be addressed early and time and budgets 
may force shcMt cuts. To ensure customer satisfaction, the courseware needs to be accq^ted by all those involved. 

**The Practitioners* Method** assumes that the client has identified the topic for training and that CBT is 
required The stages of the pnx^ess involve: 

• Minimizing the less productive parts, such as finely defmed objectives. 

• Extracting subject matter and producing narratives, not derailed design scripts. 

• Creating a prototype of the courseware for review and agreement 

• Producing intermediate versions of the coiuscware for tesung and review. 

• Systematic accq}tance testing. 

The method assumes that the design is rarely perfect first time and that change will be accommodated. Once the 
courseware is under development, changes are made in software rather than on paper - the only true documentation is 
the course itself. This presentation discusses the each stage in detail, drawing upon a number of past projects, with 
ideas for short cuts and maintaining customer approval. 

Jeff Oliver 

Castle Learning Systems Ltd 
PO Box 741 
Sible Hedingham 
Halstead 

Essex C09 3ED UK 



34 



Selected Abstracts 



COMPUTERIZED CERTIFICATION EXAMS FOR REACTOR OPERATORS 
AT THE ADVANCED TEST REACTOR 



L. W. Peterson and Lee. Ann R. Summers, EG&G Idaho Inc. 

The training dcpartmenl for Ujc Dcpanmcnl of Energy (DOE) owned Advanced Test Reactor (ATR) is rcsponsi olc for 
the certification of ATR reactor operators. The operators initial training program consists of two years of classroom 
and hands-on instruction culminating with comprehensive cognitive and operational exams. Once certified, reactor 
operators are tested annually to ensure knowledge and skills are maintained. In 1988, instructors at the ATR facility 
custom developed and hand graded over 130 written exams that met specific procedural and content guidelines. These 
former tests consisted almost entirely of essay type questions which have the advantage of evaluating higher level 
thinking, but the disadvantage of being inconsistently graded. With the expense of developing, grading, and 
completing a certification exam (about 10 hours per exam), a more cost effective and consistent method needed to be 
developed. After extensive research, the software package EXAMINER (Media Computer Entwprise) was modified, 
purchased, and installed on IBM PCs. The development of a data base structure, a system f r creating valid, reliable 
question items, and a testing prescription strategy was completed prior to entering information into the software. In 
June of 1989, ATR administered it's fully computerized comprehensive certification exam iot reactor operators with 
good success. Why the change was made, how testing requirements were met, the software features most appreciated 
by instructors and students along with other operator comments will be discussed during this presentation. 

L* W. Peterson 
EG&G Idaho Inc. 
PO Box 1625 

Idaho FaUs, ID 83415-2410 
(208) 526^179 



A REVIEW OF COOPERATIVE, COMPETITIVE, AND INDIVIDUALISTIC GOAL 

STRUCTURES AS APPLIED TO 
C0MPUTER*ASSISTED INSTRUCTION 



Gary C. Powell, University of Georgia 

The academic goals of ;ndividuals may interrelate in three ways: cooperative, competitive, and individualistic. The 
respective correlations are positive, negative, and no interdependence. Based upon the mode of interdependence in 
which interaction takes place, differing interaction patterns, motivational systems, achievement levels, and 
expectations for future achievement, will occur. Based on a literature search, fmdings and recommendations for CAI 
developers will be made. 

Gary C. Powell 
The University of Georgia 
607 Aderhold Hall 
Athens GA 30602 
(404) 542-3810 
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DEVELOPMENT OF INTEGRATED TRAINING SYSTEMS 



Michael Reakes, Hughes Training Systems 

This presentation examines the development of an Integrated Training System, defmed as a well-designed system of 
courseware, training devices, training personnel and suppoit systems, fully integrated to deliver guaranteed, qualified 
professionals in the most cost-efTicient manner. The role of a computerized management information system, termed 
Training Management System (TMS), as a tool in the management iMtxess is examined. 

Development of an integrated training system starts with a careful analysis of the job to be trained. Application of 
the techniques of Instructional Systems Design (ISD) and Systems Engineering identify the job task, subordinate 
skills and knowledge, entry behaviors and characteristics, and determine performance objectives, criterion-reference 
test-items, and formulate an appropriate instructional strategy. This process leads to the development of 
specifications for the training system, including requirements for the training media. Typical training media for a 
large training system can include lectures an<i seminars, a computer-based training system, part-task training devices, 
high-fidelity training devices, and the use of the actual equipment - for example, the aircraft for flight training. Each 
lesson should make the best use of each training device to progressively inaease the skill and proficiency level of 
each trainee to the required and guaranteed level. 

Management of the trainee, instructors, curriculum and training media in a large Integrated Training System, is 
facilitated using an automated TMS. It maintains records of training conducted, tracks the progress of individual 
trainees, and progressively confirms that they attain the knowledge, skills and task performance required to meet the 
guaranteed performance levels. The TMS identifies any deficiencies of individual students, and recommend corrective 
actk)n, such as supplementary training. The TMS allows the daily and hourly availability of trainees, instructors and 
training media to be specified, prioritizes training requirements, and performs optimal scheduling of training 
resources. The TMS allows quick rescheduling for the inevitable ^ort-notice changes due tc student or instructor 
illness, training equipment failure, and adverse weather in the case of flight training. 

The TMS also performs analyses used in tlie operational evaluation of the integrated training system. The TMS 
collects and analyzes student performance data, student throu^put arxl scheduling data, establishes meaningful trends, 
and generates rqx)rts, so that the overall training system can be improved. 

Michael Reakes 
Hughes Training Systems 
21 16 Arlington Downs Road 
Arlington, Texas 7601 1 
817-695-2020 



EVALUATION OF CBT IN ACCOUNTING 



Thomas H. Rowley, Georgia State University 

This paper describes a PC-based tutor that explains the accounting procedures used to prepare a complex financial 
statement Reported are the results of a controlled experiment to evaluate student knowledge and retention using 
pretests, posttests, and midterm exam scores. Perceived usefulness reported by students is blocked by MBTI type. 
Also discussed are sonoe CBT/ES opportunities for instructional design research in accounting. 

Thomas H. Rowley 
Georgia State University 
School of Accountancy 
Atlanta G A 30303 
(404)651-4493 
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CBT DOWNUNDER: A REPORT ON THE 1990 AUSTRALIAN CBT CONFERENCE 



Rod Sims, University of Technology, Sydney 

This presentation will idcnUfy the major findings, issues and directions from each of the thc...cs addressed during the 
CBT/90 conference, with the aim of providing a detailed perspective on the techniques used by Australian 
practitioners for effective CBT design, development and implementation. The conference featured keynote 
presentations op Future Directions in CBT and the benefits of introducing Embedded CBT systems, both of which 
emphasized the changing nature of CBT implementation concepts. The first invited paper focused on aspects of 
Metaphor-linked Authoring* and the increasing demand for user-oriented interaction; this was complemented by the 
complex demands of assessing and installing CBT hardware and courseware into the training system of an 
international airline. The major presentations were supported by presented papers which covered a range of issues 
including CBT Case Studies, Research, Computer-Managed Learning, Design & Development and CBT 
AH)lications. Of special interest and value were the outputs from the panel sessions, which have been documented in 
a small publication tilled "CBT Issues". These sessions focused on simulation, authoring, accreditation, instructional 
design and user interfacc/saeen design. Overall, the conference provided the framework for developing Australia's 
CBT industry through the implementation of innovative design techniques and alternative implementation strategies. 

Rod Sims 

Department Information Studies 
University of Technology (Kuring-Gai) 
Lindfield Australia NS 2070 
61-2413-8182 



SELECTING AN AUTHORING SYSTEM: 
TRADEOFFS, COMPROMISES, AND CONSEQUENCES 



Linda S. Stanley and James R. Ohey, University of Georgia 

Dozens of CAI authoring systems with different features and capabilities arc available for course development. 
Selecting an ^}propriaie authoring system depends on factors such as the insiniction itself, the skills and abilities of 
persons doing the authoring, and the environment in which the instruction will be used. Some authoring systems 
tradeoff power with flexibility. Some require programming ability and others arc menu-driven. Some are suited to 
one kind of instruction, but not another. And some require lengthy training while others can be used after modest 
study. 

In this paper, two factors to consider in selecting an authoring sy.stem will be examined. Two well-known, powerful, 
but highly dissimilar authoring systems (Authorware and H>^rCard) will be contrasted. The same instruction 
developed with each system will be used to demonstrate their capability. An analysis will then be made of the two 
authoring systems in terms of development time used, type of authoring and programming skills required, 
compaUbility with hardware, data-keeping capability, and ability of the systems to implement instruction in the 
intended manner. 

L.inda S. Stanley 
The University of Georgia 
607 AderiK)ld HaU 
Athens OA 30602 
(404) 5424210 
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INTEGRATING RELATIONAL DATA BASE PRINCIPLES 
INTO CBT AUTHORING ENVIRONMENTS 



Janet A. Wise, R. Ken Brownlow, and Angela G. Cobum, American Airlines 

This presentation reports the results of our investigation into a new CBT authoring environment that includes 
modular integration of: computer software applications, Instiuctional Design tools, and a relational database. 

Tools for develc^ing Computer-based Training (CBT) have evolved finom progrunming language to authoring 
systems. With the availability of new hardware and software technology, we are able to sicp beyond authoring 
systems into authoring environments. 

Janet A. Wise 
American Aiilines, Inc. 
Fli^ Academy MD871 
4601 Hwy. 360 
Fdft Worth, TX 76155 



USING AUTODESK ANIMATOR"^ IN 
COMPUTER-BASED TRAINING MATERIAL 



Julian Wakeley, Abbey National pic 

Abbey National, one of the United Kingdom's largest personal financial OTganizations, is an established user of 
Computer Based Training (CBT). It has access to over 1^00 tenninals for training purposes and the subjects covered 
vary from procedural infonnation to system simulations. 

One large programme developed in 1990 was a complete training programme for *he 1,000 Automated Teller 
Machinal (ATM) installed throughout Abbey National. Due to the security implications, and the high cost of off- 
site training with real machines it had not been possible to show trainees the inner woriungs and maintenance 
procedures of the ATM until they needed to operate the machine in a 'live* enviromnent (money loaded in ATM, on- 
line to mainframe, customers in building etc..). 

The introduction of the current generation of animation tools has given designers the ability to integrate short video 
animation clips into standard CBT programmes. 

This presentation details the process Abbey National undertook in developing short video sequences which could be 
used in CBT programmes. It will cover : 

Autodesk Animator^ overview 
Adcfitional hanlwaie itquirements 
Design considerations 

Issues regarding integration into CBT material 

This particular suite of programmes is under continual development and so the latest evaluation infonnation will be 
made available as part of the presentation. 

Julian Wakeley, Abbey National pic 
2 Dudswell Mill, Wharf Lane, Dudswell 
Beridiamsted Herts England HP4n3Y A 
908-36611x2203 
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RAPID PROTOTYPING METHODOLOGIES FOR COMPUTER BASED INSTRUCTION: 

TWO EXAMPLES 



Sherwood Wang, University of Colorado; Dana Wunderlid . McAir Corp.; and D. Owens, and UCAR-COMET 
Introduction 

Rapid prototyping is an emerging and important element in insunictional systems design today. There are several 
ways which rapid prototypes can be used in an instructional desijn setting. This presentation will highlight two 
distinct applications of rapid prototyping emphasizing the diversity of functions that prototypes can fulfill. 

The McAIr Story Board Developer 

Several computer based training programs currently being designed at McAir use a word processing program for the 
creation and revision of storyboard documents. This approach has brought to light many practical problems which 
impact both quality and productivity. In response to these problems, a structured storyboard development tool was 
created at McAir using a rapid prototyping approach. Input was elicited from the storyboard authors in the miiial 
planning stages and throughout the development of the tool through informal meetings and "wish lists". Many of 
these features were then implemented in a Macintosh environment using HyperCard. The tool was put to use as soon 
as an initial version was created and updates (31 to date) were circulated as the original program was amended. There 
was a constant flow of input from the courseware authors about the needed improvements to the tool which directed 
the type of enhancements which were implemented. 

Workstation Prototyping at UCAR<COMET 

UCAR-COMET (University Corporation for Aunospheric Research-Cooperative Program for operational 
Meteorology Educauon and Training), based in Boulder Colorado, is currently engaged in producing a set of 
computer based« interactive video modules on interpreting weather radar images and forecast methodology for several 
government agencTes. There were several circumstances unique to this project which led choosing a rapid 
prototyping approach. First, the goal of creating a realistic working environment required a complex and time 
consuming programming effort. This created problems for designers because they could not fully envision the final 
product Rapid prototyping allowed the design team to utilize an iterative process, refining their conceptions of the 
user interface and functionality without being burdened with the details involved with the final programming effort. 
The prototypes were also used to demonstrate the functionality of the final product to the sponsoring organizations 
and to facilitate communication between the designers and the programming team. Formative evaluation with typical 
users was also a problem for the aesign team because of the time lag between the lesson designs and the final 
delivery system. The rapid prototype allowed typical learners to provide feedback on the user interface before a 
permanent version wa^ created by the programming team. This prototype was implemented in a Macintosh 
environment using Supercard. 

This presentation will detail some of the design considerations which went into both prototypes and highlight the 
functional similarities and differences in the design and implementation of the prototypes as well as demonsu-ating 
the two prototypes. 

Sherwood Wang 

University of Colorado, Denver 

1200 Larimar Street 

P.O. Box 106 

Denver CO 80204-5300 

(303)744-7507 
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INTEGRATING RELATIONAL DATABASE PRINCIPLES 
INTO CBT AUTHORING ENVIRONMENTS 



Janet Wise, Ken Broy/^nlow, and Angela Cobum, American Airlines 

This presentation reports the results of our investigation into a new CBT authoring environment that includes 
modular integration of: computer software i^Iications, Instructional Design tools, and a relational database. Tools 
for developing computer-based training have evoived from programming languages to authoring systems. With the 
availability of new hardware and software technoio^^, we are able to step beyond authoring systems into authoring 
environments. 

Janet Wise 

American Airlines 

GSWFA MD 871, 4601 Hwy. 360 

R)it Worth TX 76155 

(817)967-5207 



HOW TO TALK TO A SUBJECT MATTER EXPERT 



Larry E. Wood and John M. Ford, Brigham Young University 

Using adaptations of methods from several social science disciplines as well as instructional science, we have 
developed a method for interviewing subjea-matter experts when gathering information for instructional and traimng 
systems. Our ^jproach consists of four components with guidelines for questions suited, to each component Firsts 
the descriptive eUcitation stage is intended to reveal the important entities and concepts in the donuiin as reflected in 
the terms and q)ecialized language used by the expert and provide the general structure of domain problem solving. A 
second stage, structured expansion, is designed to pr6be the attributes of domain concepts and entities, the 
relittionships between the domun concepts, and the organization of the expert's knowledge, using the terminology 
uncovered in the previous stage. The scripting stage relies on the declarative knowledge found through the two 
previous stages to discover detailed procedural knowledge (at both macro and micro levels) involved in solving 
domain problems. A fmal component, validation, is important throughout the process of knowledge gathering to 
ensure that the knowledge being elkited is contct and adec^^ purpose. 

Larry E . Wood and John M. Ford 

Brigham Young University 

SWKTBox75 

Provo,UT 84602-1 130 

(801)378-3405 

WoodL@yvax.byu.cdu 
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AN ENVIRONMENT TO LEARN ARGUMENTATION 



Andri Boder, Project Start Up 

Hypertexts can be a powerful tool in CBI. So far. most learning environments created with this tool have been built 
in the spirit of electronic books. Some didactic methodology should be provided along with the database. Very often 
it is not the case. The reason is probably that these environments include clear navigation devices. 

The environment presented here is an example that, in addition to their strength as a flexible and very interactive 
database, hypertext tools can assist in the thou^t process brought to bear by the student in a situation of knowledge 
acquisition. 

The environment is a simulation of the defence of a law case. In the process of finding an approfniatc defence plan, 
strategies, hypothesis, comparisons, analogies, classifications and other conceptual analysis are developed by the 
studcnis. The environment makes full use of resources provided within CBI to assist in these thought processes. It 
also shows how ccwitrol can adequately be passed from and to the machine whenever feedback is needed Also strong 
emphasis is put on making links between different domains of knowledge involved Qog}C^ law, third-world issues, 
human relations). The environment is designed in such a way that both top-down and bottom-up reasoning are 
induced. 

Andr6 Boder Projea Start-Up 

RO. Box 172 

121 1 Geneva 3 Switzerland 

4122-78M530 



LOGO INSTRUCTION AND STUDENTS ORDERED TREES 
OF PROGRAMMING CONCEPTS 



Gref^g Brownell Bowling Green State University; and Dieter Hrkler, Mead Corporation 

This study examined the development of mental models of Logo iKogramming concepts in 17 5th through 7th grade 
stL^dents. Using the ordered tree technique, measurements of student's menial models were obtained in the first, third, 
and fifth weeks of a five week computer camp. Student's understanding of basic Logo commands was measured at the 
same intervals. Student's mental models of Logo keywords developed significantly (ii<.001) in organization during 
the camp. Development in the organization of student's mental models was significantly (b<.05) related to student's 
understanding of Logo keywords. This suggests that small amounts of instruction in piogramming can have an 
impact on student cognition. Changes in student's representation of programming concep'tS m^y be the lirst step in 
developing programming and general reasoning skills. If, hov.ever, students spend a lor^ tiir e developing accurate 
models of a programming language, they may have limited time to develop higher-order reasoning skills. 
Educational programming environments that minimize the differences between mental models of novice student 
programmers and the conceptual models underlying educational programming environments would seem to be a 
desirable aim for software developers. 

Gregg Brownell 

Bowling Green State University 
Department ofHDCI 
Bowling Green KY 43403 
(419) 372-7359 
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CAI DESIGN FOR METACOGNITiON 



Bernard J. Dodge, San Diego State University 

Recent research has supported the iniportance of teaching learners to become aware of their own cognitive processes. 
The design of a project to leach metacomprehension with a simulation game for at risk high school students will be 
described. 

Bernard J. Dodge 

San Diego State University 

Educational Technology Department 

San Diego CA 92182-0311 

(619)594-7401 



A MEDIA CENTER FOR THE 21 ST CENTURY 



VickiDurbin and Donna Schell Yavapai School 

The onset of the Infonnation Age brings unique challenges to schools as we grapple to keep up with the vast 
amount of infonnation available. Yavapai School has met this challenge with the opening of a new, state-of-the-art^ 
technological media center. It is a media center geared to allow our school community access to infonnation and to 
ap0y it in meaningful and knowledgeable ways. An array of services incorporating computers in nearly every facet 
of operation and most learning situations make Yavapai a truly futuristic school. 

Vicki Durbin 

Yawapsi School 

701 NcmhMUler Road 

ScottsdaIeAZ852S7 

(502)423-3250 



COMPUTERS, CRITICAL THINKING, AND FRESHMAN COMPOSITION 



Rex Easley and Gary Vaughn, University of Cincinnati 

The use of computers in the fireslunan composition (writing) classroom opens up a number of possibilities beyond 
w(Kd-processing and text revision activities. Some software, such as Writer's Helper by William Wresch, offers 
writing assistance in the areas of generating and organizing ideas as well as analyzing texts for readabUity level and 
other **mechanical** aspects. Writer's Helper can also be used as the basis for assignments which promote critical 
thinking skills, creativity in writing, and development of an authorial voice appropriate to a number of writing 
situations. 

In this {presentation we will discuss the use of the computer and the Writer's Helper software, in conjunction with 
word processing capabilities in several writing assignment projects which promote critical thinlcing skills by 
requiring students to generate ideas, make chCHces. and follow the implications of those choices in the planjiing and 
writing of an extended essay. The assignments are based on a creative writing approach to reading and writing about 
shon stories. They rely on reading and predicting, personal response to what is being read, short writing assignments 
that ask for creative writing responses to situations, and a collaborative project that lets students resolve writers' 
dilemmas. All of the reading and writing assignments use Writer's Helper in a number of ways to facilitate the 
students' work on their writing. 

Rex Easley and Gary Vaughn 
University of Cincinnati 
M.L. #203 Cincinnati. OH 45221 
513-556-1794 
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THE COMPUTERIZED WRITING CLASSROOM 
AND "HIGH RISK" COLLEGE LEARNERS 



Rex Easley and Gary Vaughn, University of Cincinnati 

Junior colleges across the country are confronting the challenge of how to attract and retain adult returning students. 
As we move into the I990's, access to education for non-traditional students can be enhanced by the computerized 
classroom. 

In the two-year, open admissions University College of the University of Cincinnati, writing instructors are in the 
fifth year of a continuing study of the effects of word processing and critical thinking software on the composing 
processes and attitudes toward writing of developmental writers. Most students in the basic writing class (English for 
Effective Communication) are recent high school graduates; however, many are admitted through the Pre-Technical 
Training Program, a course of study funded by the Job Training Partnership Act and designed to attract older, "high 
risk" students to the college environment. The "average" Pre-Tech student is highly motivated, but her ambitious 
goals arc often hindered by a lack of comfort with her academic abilities. The research study at UC divides the basic 
writing classes into sections that use a simple word processor, sections that combine the word processor with 
thinking- aid software, and classes mat do not use computers at all. Our non-traditional Pre-Tech students in the 
computer environment perform better on a holistically graded final exam than do their traditional counterparts. We 
believe the adult learners perform better when working with computers because this type of learning environment 
assists the adult learning style outlined by Malcolm Knowles, Stephen Brookfield, and Paul Wcstmeyer. The 
computer allows such students a "co-directed" learning environment where they are free to lake risks and experiment 
with language. In addition to improving their writing, they appear to improve their attitude toward writing, as 
revealed by an end-of-quarter questionnaire. Our research suggests that computers and carefully chosen software can 
help bridge the gap between "high risk" adult learners and a college education. 

Rex Easley and Gary Vaughn 
University of Cincinnati 
M.L. #203 Cincinnati, OH 45221 
513-556-1794 



COMPUTER-ASSISTED INSTRUCTION ON THE MAC II 



Mary M. Finnen, Steven M. Lawton, Bryce Bate. Kathleen Davey, William Jensen, 
and Russell Skavaril. The Ohio State University 

The Mac II computer and the authoring software AuthorWare Professional has provided an effective platform for 
designing general biology instructional materials. The program. Encounters with Biology, engages the student in a 
question-and-answcr session that assists the student in discovering why biological terms and concepts are important. 
The program makes effective use of hi-resolution color images, audio, and discovery-learning techniques. 

Mary M. Finnen 

The Ohio State University 

103 GcTicral Biology Annex 

1791 Neil Avenue 

Columbus OH 43210 

(614)442-0701 
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HISPANIC MATH PROJECT: A MULTIMEDIA LEARNING STATION 



Wayne Hartschuh and Rocio Sanchez, Arizona State University 

The Arizona State University Hispanic Math Project (HMP), sponsored by the National Science Foundation, is 
developing an intelligent, computer-based interactive video learning station capable of adaptively diagnosing and 
individually tutoring sixth-grade migrant Hispanic students. The learning station is coordinated by an expert system 
which individualizes instruction with regard to content, language and delivery medium, provides empirical data about 
the interaction of these factors and evaluates them in order to better integrate these domains during the system's 
interaction with the student 

The major goal of the Hispanic Math Project is to [H'omote the achievement of migrant Hispanic elementary students 
in mathematics - focusing specifically on the mathematics of measurement. A second major goal is to explore the 
effectiveness of individualized computer technology to complement classroom teachers in addressing the critical 
elements in migrant Hispanic education. 

The presentation will focus on the development of the liMP Learning Station and its implementation in the schools. 
The research data from the project will provide educators with information that can be used to distinguish between 
math difficulties and language barriers so that future instruction may take into account the interaction of these 
variables. 

Wayne Hartschuh and Rocio Sanchez 
Arizona State University 
Technology-Based Learning and Research 
Community Services Center at Papago Pailc 
Tempe, Arizona 85287-0908 
(602) 965-3322 



FACTORS INFLUENCING THE ADOPTION 
AND IMPLEMENTATION OF AN INNOVATION 



Frances Hensley, The University of Georgia 

Schools have been slow to adopt and implement computer-based educational innovations, including computer 
assisted instruction. Whereas, generally, the most attention in the development and implementation of an innovation 
is paid to the technical aspects of the innovation, such as its goals, objectives, materials, and means, and while this 
dimension is critical, it alone can not explain the success or failure of an innovation in a school setting. In an 
attempt to better understand the factors which influence the adoption and implementation of an innovation into a 
school setting, a multidimensional model is oflered. 

Based on a review of the literature on school change, this model includes four dimensions: the technological, 
political, cultural, and psychological. The use of this model allows for the examination and consideration of various 
dimensions of an educational innovation and how these dimensions impact and interact with one another during the 
innovation's development, implementation, and evaluation. This presentation will discuss each of these dimensions 
and advocate for the need to examine educational innovations through the use of a multidimensional model which 
allows the consideration of the richness and complexity of both the innovation and the environment in which it 
exists. 

Frances Hensley 

The University of Georgia 

Department of Instructional Technology 

607AderholdHaU 

Athens, GA 30602 

404-542-4038 
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PERSONAL AND ACADEMIC APPLICATIONS OF HYPERCARD: 
A HIGH SCHOOL COURSE 



Annette Lamb, Uni-^ersity of Toledo 

HyperCard has become a popular tool for educators. Because of its ease-of-use, many educators are recommending it 
as an excellent tool for students. This presentation describes the use of this tool by high school students within a 
high school computer applications course. A course outline and sample class activities are discussed Class activities 
include information presentation and exploration, interactive stories and reports, note taking systems, and surveys. In 
addition, the use of emerging technologies such as scanning and CD-ROM technologies are discussed within the 
context of practic::!, personal and academic student projects. 

Annette Lamb 

The University of Toledo 

2801 West Bancroft, 2021 GH 

Toledo, OH 43606 

(419) 537^713 



MANAGING THE COMPUTER-BASED CLASSROOM 



Frederick Knirk and Mary Pinola; University of Southern California 

Teachers at all levels of instruction are challenged to manage the computer-based classroom. Management is critical 
as students have increased access to computers in and out of the classroom. This presentation will describe strategies 
for: 1) moniu>ring the application of computers to instructional tasks and for conducting on-going evaluation of 
student response and teaching systems; 2) promoting active learning; 3) selecting ^ropriate media and instructional 
methods; and 4) responding to accountability requirements. 

Mary Pinola 

University of Southern California 
330 St Katherine Drive 
FlintridgeCA 91011 
(818)790^33 
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COMPUTERS IN EDUCATION: A GRADUATE PROGRAM DESIGN 



Lynne Levy and Louis Molinari, Glassboro State College 

The unique post-baccalaurcate program, Computers in Education^ is a certificate program at Glassboro Slate College, 
Glassboro, New Jersey. A major goal in this design is to encourage seasoned educators to incorporate the computer 
in their classroom so that articulation between technology and regular classroom activity can become a working 
reality. 

This iS'Semestcr hour graduate program provides participants with a comprehensive knowledge of computer use in 
the educational setting. The program helps the participants to become competent in computer utilization for 
instructional, administrative, as well as personal use. The program can be completed as a stand-alone certificate 
program, or it can be incorporated into a master's degree emphasis area; thus affording teachers the opportunity to 
continue their professional development while acquiring a new area of expertise. 

Elementary and secondary teachers are serviced by this program, along with provisions for supervisors and school 
administrators. Instruction is delivered in both on-campus and off-campus settings. Instruction utilizes both the IBM 
and/or Apple systems depending on district needs and specific course requirements. 

This program is the primary training ground in computer-related instruction for teachers in Southern New Jersey. 
Modifications are made in the design to accommodate district needs and to provide teachers with new developments in 
classroom computer applications. 

Lynne Levy,Ed.D. 
Louis Molinari, Ed.D. 
Glassboro State College 
Glassboro, New Jersey 08028 
(609)863-7042 



WHICH PARENTS SEND THEIR YOUNG ONES 
TO COMPUTER CLASSES AND WHY 



Charlotte Scherer, Bowling Green State University 

Parents who enrolled their four and five year old children in a computer class on the BGSU campus completed a 
questionnaire designed to gather demographic data and information about their reasons for putting their children in the 
class. Their general attitudes about the importance of computer competence were also surveyed. Since this is the 
only computer class of several children's o^mngs which has consistently been filled to overflowing, the investigator 
was curious about its extreme popularity. Did parents believe that an early start was necessary for later success in 
school? Did parents believe that young children could use computers to learn numeric and alphabetic skills? Did they 
think that computers were fun for preschoolers? What was the parents' educational level? Such was the nature of the 
questions asked in this survey. 

Results indicated these highly educated parents in a college community thought their children would have fun and 
enjoy learning to use a computer, but follow-up phone calls to 14 parents indicated they also felt it was important, 
with computers so widely used, that their child know how to use computer as soon as possible. They also felt that 
computers could help their child learn basic skills. 

Giarlotte Scherer 

College of Education & Allied Professions 
Bowling Green State University 
Bowling OH 43403-0243 
(419) 372-7392 
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THE TALIESIN PROJECT: 
INTEGRATED CURRICULUM USING HYPERMEDIA 



Eric E. Smith, University of Northern Colorado; and John Redman, Hewlett-Packard Company 

The first year of the Taliesin Project, its theoretical foundations, the development of Pilot materials, and the 
approach to meeting the need of improving the mathematics, science, and technology skills and knowledge of 
students in K-12 education in the United States are discussed 

At the root of the project are the beliefs that teachers play pivotal roles in the education of children and that children 
can and will learn about mathematics, science, and technology if their interest can be captured. Relevance to them is 
shown, and their confidence with the content in increased. 

To meet these concerns, it is necessary to provide teachers with content and materials that can be made interesting 
and relevant This project uses richly textured initiating media (e.g. movies, books, athletic events) that illustrate 
technological or natural systems from which scientific principles and mathematical concepts are derived. By using 
hypermedia, access to multiple paths through the material are available to met both student and teacher needs and 
preferences. While the system supports almost any instructional model, the materials under development follow 
Elaboration Theory and Cooperative Learning paradigms. 

The Taliesin Project is a cooperative project of the University of Northern Colorado, Hewlett-Packard Company, and 
several school districts in the state of Colorado. 

Eric E. Smith 

University of Northern Colorado 

Division of Research Evaluation and Development 

Greeley CO 80639 

(303)351-1674 



FRENCH AND GERMAN DICTION: 
DESIGNING AN ADAPTABLE HYPERCARD STACK 



James T. Talley and Donna £. Williams, The Ohio State University 

Since creating courseware consumes a considerable amount of time, it is essential to conserve development time 
wherever possible. One means of time savings is to survey existing courseware before allocating resources for 
vclopmenL Another is to adapt existing courseware to fit the needs of multiple users. Our presentation will focus 
on the development of a HyperCard stack that has been adapted for use in two courses. 

James T. Tallcy 

The Ohio State University 

OTBLordHaU 

124 West 1 7th Avenue 

Columbus OH 43210 

(614) 292-9821 
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INSTRUCTIONAL TRANSFORMATION THROUGH 
INSTRUCTIONAL TECHNOLOGY: A MODEL FOR CHANGE 



Robert Wambaugh, Pennsylvania State University 

The National Task Foice on Instructional Technology (1986) called for the educational systems of the United States 
to undergo imniediate reformation and work toward eventual transformation in order to take advantage of the potential 
of new instructional technologies. The Penn State Regional Computer Resource Center is pressing a five step 
model for introducing techncriogy into the schools suggesting a hierarchy of steps that must be worked through if the 
tools of technology are to be appropriately ^plied to education. 

Robert Wambaugh 
Penn State University . 
202 Chambers Building 
University Park, PA 16802 



MOVING STAND-ALONE COURSEWARE INTO A MANAGED SYSTEM 



Paul Wieser and Eileen Wilkie, MECC 

MECC Management Master (MMM) is a courseware managennent system that delivers lessons based on MECC 
software, records performance data» and rqxMts that data. It was developed to assist teachers who want to integrate 
MECC reading azid math software into their curricula, yet may not have the time to correlate the software to their 
objectives or track student performance in order to insure that students receive appropriate instruction. 

The instructional software managed by MMM was originally developed to stand alone, delivering instruction 
addressing discreet instructional objectives* Moving this software into a managed system that is curriculum-based 
poses two major challenges. First, the software must be analyzed for instructional objectives and a means of 
assessing student performance on the objectives must be developed. In some cases, the stand-alone courseware needs 
few modifications in order to fit into the system, but in other cases significant changes must be designed and 
implemented. The second major challenge comes in designing the correlation process: analyzing textbooks, 
establishing rules for when correlations will be made between textbook lessons and MECC software and deciding 
bow program paranoeters will be set for given lessons or groups of lessons. 

Paul Wieser 

Minnesota Educational Computing Corporation 
1769 Hubbard Avenue 
St Paul MN 55401 
(612)481-3553 
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THE EFFECTS OF ACCESS TO SIMULTANEOUS INTEGRATED 
INFORMATION SYSTEMS ON LEARNING 



Ruth Curtis and Barbara L Grabowski, Syracuse University 

With the rapid development of interactive learning systems, vast amounts of information can be presented 
simultaneously through the integration of text, graphics, audio, and video formats. Research that deHnes the 
characteristics and relationships of and differences between information-based systems and instructional technology 
systems will lead to understanding their capabilities and develc^ing appropriate ^plications which facilitate learning 

Using an intensive design ^proach, this exploratory study investigated ways in which the users* prior knowledge, 
motivation, and cognitive processing influenced the way in which they accessed and processed information through 
interactive systems. Six entering freshmen (two males and four females) were observed as they explored the ABCs 
Election '88 Interactive Videodisc. Firsts Witkin's Group Embedded Figmes Test was administered to dcicrmine their 
level of fleld dependence/independence. Prior to and after exploring the Election *88 disc, they were asked to draw a 
pattern note to assess their prior and post organization of knowledge, and were administered an assessment using 
Keller's (1986) ARCS categories to determine their level of motivation in the *88 elections and the interactive video 
technology. 

In an expkxatory way, both qualitative and quantitative data were used to examine the results. Qualitative methods 
were used to analyze the phenomenon of the complex interrelationships and dynamic nature of cognitive processes 
involved in interactive information retrieval. This method enabled us to examine natural interactions operating 
among variables in real learning situations (Vacca and Vacca, 1980). Quantitative methods were used to determine 
the effects of various navigational patterns on achievement and motivation. Results will be presented which 
interrelate the qi:^^*tative and quantitative findings. 

References 

Keller. J.M. (1986). Motivational Design of Instruction. In CM. Reigcluth (ed.). Instructional Design Theories a*ul 

Models: An Overview of Their Current Status. New York: Erlbaum. 
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SNOWMAN: A SYSTEM FOR KNOWLEDGE MANAGEMENT 



John Af. Ford and Larry £. Wood, Brigham Young University 

Wchavc developed a prototype "knowledge editor** for documenting and managing information gathered from experts 
during development of instructional and training systems. It is designed around the distinction between declarative 
knowledge (domain concepts and objects and their relationships) and procedural knowledge (the use of declarative 
knowledge in solving domain problems). For declarative knowledge, the tool allows the user to enter concepts and 
their relationships, display individual concepts and their relationships with others, and to graphically display 
relationships among concepts in a semantic network. For representing procedural knowledge, the tool provides the 
capability for using structured outlines, the components of which can be easily inserted into a free-form text field. 
As an editor, the tool allows for revision and updating as additional information is gathered. Furthermore, it provides 
the capability to represent knowledge in a form intermediate between original sources and final product This makes 
it possible to verify and validate it prior to a "production"* stage. The prototype was developed on a Macintosh 
computer in SupeiCard. 
HERE 

John M. Ford and Larry Wood 
Brigham Young University 
SWKTBox 75 
Provo,UT 84602-1 130 
(801)378-3405 
WoodL@yvax.byu.edu 



ADDING A HYPERMEDIA INTERFACE AND HELP SYSTEM 
TO AN EXPERT SYSTEM ADVISOR 



R. Scott Grabinger and David //. Jonassen, University of Colorado at Denvt*' 

A useful application of an expert system is as an intelligent job aid for facilitating the design and development of 
instruction employing the instructional systems design process. The IDD Advisor is an expert system for facilitating 
the design and development of instructional sequences. The IDD Advisor employs two major emerging technologies: 
(1) an expert system using a structured rule base; and (2) a hypertext front end and support system containing the 
user interface to the expert system and the conceptual knowledge for understanding. Specifically, the IDD Advisor 
provides consultation on the selection of an instructional tactic for a given insuiictioiia! situation described by the 
user. However, advice is useful only if it can be applied correctly to the instructional situation. Therefore, an 
advisor needs to provide backgroimd conceptual information along with the rules that help the novice make 
decisions. The solution that we have adopted is to integrate a hypertext-based user interface and online help system 
to facilitate access to the information and provide the conceptual knowledge needed to understand the all of the 
terminology needed to make decisions as well as the results of the consultation. The IDD Advisor demonsuated at 
ADCIS is a combination of the the Levels expert system and a hypertext system produced in HyperCard. 

R. Scott Grabinger and David H. Jonasscn 
University of O)lorado at Denver 
Campus Box 106 
P.O. Box 173364 
Denver, CO 80217-3364 
(303) 5564364 
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EMERGING TECHNOLOGIES AND LEARNING ENVIRONMENTS: 
FROM INSTRUCTION TO LEARNER CENTERED MODELS 



Michael J. Hanru^i and James Gall, Florida State University 

During the past 20-30 years, interest in computer-based instruction (CBI) has grown dramatically. Enhanced power, 
increased availability of peripheral devices, and developments in hypermedia have created extraordinary potential for 
CBI design. Likewise, recent developments in expert systems, artiHcial intelligence, and knowledge engineering 
have yielded interesting perspectives on software design, with considerable long-term promise. At the same time, 
there h^ve been significant, though largely unexploitcd, advances in cognitive psychology, situated cognition, and 
constructivism. Collectively, these advances offer the potential to redefine learner-computer interaction. 

Yet, in many ways, our designs have failed to evolve accordingly. Most efforts based upon contemporary 
psychological and technological developments have emphasized new ways for the computer to organize instruction 
rather than new ways to empower students to learn. In effect, we invent new ways to do old things, but not new 
things. We work to optimize the capabilities of the technology by emphasizing externally centered adaptation, but 
often fail to break new ground in learner-centered designs. 

How can the potential of emerging technologies, and related developments in cognitive psychology, be combined to 
yield fundamentally different learning experience.^'/ One possibility lies in the growth of interest in learning 
environments. Learning environments are problem-/;entered, multi-dimensional systems designed to provide highly 
contextualized learning opportunities. The proUems typically require resolution by the student of some 
inconsistency, problem, or crisis through guided exploration among related components. Components such as a 
knowledge base, pedagogue, linkways to related concepts, and so forth, are organized interactively to reflect 
concurrently the interdependence among related knowledge and skills. Students maneuver through the environment 
using readily accessed tools and system utilities which permit interaction at multiple levels (e.g., navigation, 
manipulation, hypothesis testing, etc). In this presentation, the rationale for, and attributes of, learning 
environments will be presented, m addition, working prototypes of learning environments will be demonstrated, and 
audience participation will be solicited. 

Michael J. Hannafin 
Florida State University 
CIDS, 2003 Apalachce Pkwy. 
Tallahassee FL 32301 
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INTEGRATING HYPERMEDIA AND KNOWLEDGE-BAS^D SYSTEM TECHNOLOGIES 
FOR TRAINING AND INFORMATION DISSEMINATION 



Francis A. Harvey, Lehigh University 

The goal of the Combined Advanced Infonnaiion Technology Systems (CAITS) Project at Lehigh University's 
Engineering Research Center for Advanced Technology for Large Structural Systems (ATLSS) is to utilize 
traditional database systems, hypermedia, and expert systems technology to develop intelligent hypermedia-based 
infonnation systems. Systems developed will be used for the support of engineering research, for training on ATLSS 
systems, and for transfer and dissemination of the results of ATLSS projects. 

Systems developed use a synergistic combination of powerful database management software (for information storage 
and retrieval), knowledge-based systems (to support user's formulation of sear*^ <;trategies, evaluation of search 
results, and utilization of the information provided by the information system), and hypermedia (to provide an 
intuitive, easy, and efficient user intcxbcc to the overall system). Specific development techniques currently in use or 
being planned include the use of knowledge engineering techniques developed for expert systems to optimize the use 
of subject matter experts in designing and developing hypermedia materials and the use of hypemnedia to provide 
graphical navigation and browsing support for users of existing intelligent information systems. This paper will 
describe the systems under development and will discuss how the techniques and procedures developed can be implied 
to information/communications needs in other domains. 

Francis A. Harvey 
Lehigh University 
College of Education 
1 1 1 Research Drive, MTC 
Bethlehem, PA 180154793 
(215)758-3268 



NEW DIRECTtONS IN HUMAN FACTORS FOR INTIIRFACE DESIGN 



Hueyching Janice Jih and William //. Wheeler, Jr., University of Georgia 

Courseware designers are bombarded with promise of authoring tools, especially authoring systems as HypciCard, 
Linkway, Course of Acdon, or ProPi. Most of designers arc lulled to believed that a wonderful authoring system sill 
guarantee them a high-quality program. Very 'ittle is mentioned about the interface design consideration which is an 
essential authoring skills needed to create good interactive courseware. 

The interaction between a computer-based instructional system and its learners can be influenced by both the system 
itself and the engagement of the learners, such as reflecting, thoughtful, visceral, and changing, through learner- 
computer interfaces. In order to increase the learner's satisfaction and performance and to create high-quality 
interactive courseware, designers must take human factors in leamer-coursewarc interacdon into account with specific 
reference to perceptual, physiological, and psychological aspects. This presentation discusses the human factors 
consideration as a new direction for computer-based courseware design. Objectives of this presentation include: 
introduction to human factors issues in interaction; the importance of human factors for courseware designers; and 
design guidelines for Computer Assisted Instruction (CAI) or Compuicr-Based Training (CBT) programs. The design 
guidelines forms the architecture of this presentation. 

Hueyching Janice Jih 

The University of Georgia 

Department of Insuwtional Technology 

607 AdciholdHaU 

Athens, G A 30602 
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THE PHYSICS TUTOR: BUILDING AN INTELLIGENT TUTOR FROM HYPERTEXT AND 

EXPERT SYSTEMS 



David //. Jonassen and Sherwood Wang, University of Colorado 

This presentation describes the construction of an intelligent tutor from hypertext and expert systems. The expert 
model in this prototype is implicit in the hypertext structure and represented in expert system ru?r bases. The student 
model, a combination of declarative, structural and procedural knowledge, is constructed in expen system rule bases. 
The tutor will be built in expert systems also. This prototype is built in HyperCard and Level 5 on the Macintosh. 
It is designed to demonstrate intelligent functionality on a perianal computer using existing software. 

Introduction 

Intelligent tutors are expensive to develop because of the knowledge engineering and programming requirements, and 
they are expensive to implement because of the iromputation^ requirements. This paper presents a prototype 
intelligent tutor constructed with hypertext and expert systems. The combination provides most, if not all, of the 
required functionality of an intelligent tutoring system. 

Intelligent tutors provide a meaningful option to traditional CBI. They are a knowledge based foim of intelligent 
computer based instruction, normally consisting of a model of the expert's knowledge, a student diagnosis module, 
an instructional environment, and a human*computer interface (frequently a natural language interface). The 
knowledge model is typically based upon a semantic network, consisting of objects, [mperties, or concepts which 
possess names, values, or tags for accessing the object Of all of the intelligent tutoring systems (ITS) reviewed by 
Wenger (1987) and Poison and Richardson (1988), none run on personal computers, and all have required large 
development teams to produce nitors on limited topics. If ITS is to impact education, they must be developed for 
popular PCs using available software. 

Objective 

Our goal is to produce and demonstrate a prototype ITS program using an integrated system of hypertext and expert 
systems. We intend to use commercial software and Macintosh computers to demonstrate the feasibility of a generic 
rrS system. 

Jonassen, Grabinger, and Wang (1990) demonstrated different applications of the integration of hypertext and expert 
systems. Among these, expert systems may function as diagnostic tools for accessing t»»e hypertext We 
demonstrated a preliminary prototype of an ITS. The only student modelling in our preliminary prototype was 
programmed in Hyper. Talk. We are continuing this work by developing student modelling rule bases to analyze 
learning styles and prior knowledge and to assess knowledge acquisition during instruction. We hq)e to tie the 
assessment knowledge base to the expert's knowledge base in ordo- to prescribe the most productive learning palhis. 
We demonstrated those learning paths in our presentation. 

Our general hypothesis is that hypertext and expert systems provide most, if not all, of the functionality of an ITS A 
well designed hypertext interface can supplant most of the need for natural language. As an instructional 
environment, hypertext can explicitly model the expert's knowledge structure as well as providing sophisticated 
instructional transactions that elicit any type of information processing desired (Jonassen, 1989; Jonassen & Mandl, 
1990). Expert system knowledge bases can model both the expert's and novice's knowledge structure and prescribe 
appropriate instructional sequences. We are implementing this model in the form of a Physics Tutcw. 

The Physics Tutor is a prototype instructional physics information base designed to familiarize students with the 
basic concepts in rectilinear Newtonian physics. It is written in HyperCard and utilizes node and link structure to 
organize information contained in the program. The program also utilizes an expert system shell to analyze student 
responses and to provide models of student understanding. 

Interactivity is emphasized in ihe program at several levels. The system allows for browsing using a graphical 
overview, which shows the relationships between the concepts spatially. There are also two modes of structured 
browsing in the system. The user interface is structured so that the learner may both request information from the 
system or be quizzed on structural (x procedural knowledge contained in the system (Carbonell, 1970) . 

The expert system shell. Level 5, functions transparently behind the HyperCard stack, to organize responses to users' 
questions regarding relatcdness of concepts in the information base. It also navigates the student through a series of 



ERLC 



SIGHT 



nodes to show the relationship between concepts. The expert system will also be used to create a model of the 
lean.cr*s understanding of the relationship between concepts and prescribe sequences of nodes to correct the student 
model if necessary. 
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DESKTOP PRESENTATIONS: TIPS FOR USING 
HYPERCARD AS A PRESENTATION TOOL 



Annette Lamb, University of Toledo 

EducaUHS at all levels are using HyperCard as a presentation tool. Unfortunately, many users are not following 
guidelines for the development of effective presentations. This presentation discusses the design and development of 
effective desktop presentations including selection of content, screen design and layout, typography, and use of 
gr2q}hics. It provides ideas for large group, linear presentations, and ''kiosk-type" presentations. It also discusses the 
development of complex presentations that make use of HyperCard's branching capabilities. 

Easy methods for scripting presentations are highlighted, in addition to the use of presentation templates. Ideas for 
creating graphics and scanning images are also discussed. Innally, the use of emerging technologies such as compact 
discs and laser discs is discussed 

The presentation concludes with some general information regarding the delivery of desktop presentations including 
equipment and the use of supplemental handouts. 
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INTEGRATED PERFORMANCE SUPPORT SYSTEMS 



Kathleen Long, Jan Hermanns, and Clarence J. Ross, Anderson Consulting 

There was a time when most training was conducted in the classroom and via one-or.-one coaching. More recently, 
much of that training has migrated to self-study alternatives such as workbooks and computer-based training, 
employing a variety of supporting media* 

Today there is tremendous emphasis on building a competitive workforce, particularly in the use of computer 
systems and applications. It is no longer pertinent to view training as an event, apart from the job. Rather, it must 
come in many forms as part of an Integrated Performance Support System (IPSS), which can be accessed in relevant 
bits and pieces during job performance. The goal is to provide exactly what the learner needs, as quickly and easily as 
possible, at the right level of depth, with minimal disruption to job performance. 

This presentation focuses on what an IPSS is and how it can provide more efficient and effective training and support 
than more traditional methods. The various components of an IPSS will be illustrated and discussed: Job competency 
diagnosis and prescription; Multi-level helps; Job aids; On-line and paper-based reference; On-line and off-line 
training; Knowledge-based advisors; and Update bulletins. 

Kathleen Long 
Anderson Consulting 
10075 Carroll Canyon Road 
San Diego CA 92131 
(619) 578-1700 



FORMAL INITIAL STEPS IN THE DEVELOPMENT 
OF EXPERT SYSTEM COURSEWARE 



Brother Matthew Michelini, Manhattan College 

Classnxxn teachers dcvclc^ and use a wide variety of courseware materials to enhance learner understanding. Many of 
these courseware materials involve some form of learner self-evaluation through the use of multiple choice 
questions. With some modification to the program design these self-evaluation materials can offer the learner greater 
latitude in receiving more detailed information concerning both correct and incorrect responses. 

Articles and presentations concerning interactive and expert system courseware have demonstrated the interactive 
aspects of the courseware while little focus has been given to the design and implementation steps necessary for 
making the courseware more interactive and Expert System in nature. The objective of this session is to: 

1. demonstrate the interactive nature of courseware in presenting self -evaluation questions, 

2. demonstrate how the learner has access to additional information concerning the answers to the self- 
evaluation questions, 

3. illustrate how a BASIC language program can be altered to allow the programmer to incorporate more 
interactive and Expert System oriented programming into self-evaluation courseware. 

The presentation will consist of a computer demonstration of the courseware both in the old and revised formats, a 
computer printout of program statements used in the production of the courseware, and an overhead transparency 
presentation of what modifications have been made to make the courseware more interactive and Expert System in 
nature. 

Brother Mauhew Michelini 
Manhattan College 
5050 Fieldston Road 
Bronx NY 10471 
(212)920-0289 
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NETWORKS FOR LEARNING IN GROUPS: 
RESULTS WITH HIGH-RISK HIGH SCHOOLERS 



Steven L, Robinson, Wilder Research Center 

Research in teaching reading competency skills to dropout prone high schoolers was conducted over a one-year 
period. A technology-based netwoiked system was used to deliver instruction in groups of 15-23 students. This 
session will describe the technology system, development of matmal, research design and results. 

Steven L. Robinson 
Wilder Research Center 
1295 Banoana Blvd. Suite 210 
St Paul, MN 55108 
(612)647-4625 



SPECIAL EDUCATION TECHNOLOGY RESOURCE CENTER: A REPORT ON THE USE 

OF CBI TO TRAIN SPECIAL EDUCATORS 



Marshall G. Jones, Gary £. Ross, and John Langone, University of Georgia 

This presentation will focus on the progress of I Special Education Technology Resource Center (SETRQ at the 
University of Georgia. A report will be given on the development of a new interdisciplinary component to the 
masttt-'s degree program that will train special education technologists. Computer-based instruction modules, 
developed primarily in HyperCard, to train special educators will be demonstrated. Many of these modules will be 
made available as part of tii^ demonstration. 

Marshall G. Jones 

The University of GecR'gia 

Dq)artment of Instructional Technology 

607 Adeihold Hall 

Athens, G A 30602 

404-542-4770 
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EMULATED ORAL EXAMINATIONS: EXPERIENCES AND PROJECTIONS 



Michael Anbar and Ada Anbar, Stau University of New York at Buffdb 

The efficacy of computerized open-ended examinations in assessing knowledge and reasoning ability was compared 
with that of multiple choice questions answered by the same students in the same session, and with their 
perf(mumce in the same course assessed by sh(»t essay answers and term papers. The emulated oral examinations 
differentiated better than the levels of the students, and revealed substantially more about their mastery of the 
material. 



NUTRISMART: AN INTELLIGENT NUTRITION ANALYST AND ADVISOR 
FOR HEALTH CARE PROFESSIONALS 



Wm. J. Fetter and Saroj M. Bahl, University cf Texas, Health Science Center 

Dietetic programs include clinical nutrition as an integral component of their curriculum. The dietetic student has to 
acquire a considerable amount of infonnation then synthesize and apply the pertinent knowledge in the creation of an 
individualized nutritional care plan (SOAP note). This progression from theory to practice is, however, a difficult 
transition. Fuxthermorc, rigid time constrants placed on studcnis' learning makes this transition even rnoie difficult 
and anxiety-provoking. Clinical case studies or patient simulations that are designed fcM" computer delivery provide an 
excellent self-instructional, self-paced approach for presentation of nutritional care principles lo a dietetic student 

A Neural Network mimics the way we learn and make decisions, hence can function as an "expert" in this setting. 
This program combines a Computer-Based Patient Simulation with a dynamic Neural Network knowledge base. The 
student interacts with the simulation to learn as well as practice with the concepts necessary and preliminary to the 
development of a nutritional caie plan. After completion of the SOAP note, the student consults with the Neural 
Network Expert GNA) on the accuracy of the plan. Apprt^Miate case studies in various content areas - cardiovascular 
disease, gastroenterology, diabetes, cancer and renal disease are included in the program. 



Michad Anbar 
SUNYBuffido 



Bx)physk:al Sciences 
118 Gary HaU 
Buf&k> NY 14214 
(716)831-2328 



William. J. Fetter 

University of Texas, Health Science Center 
P.O. Box 301187 
Houston TX 77230-1187 
(713)792-4466 
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ANAEROBE DETECTIVE: 
AN INTELLIGENT SIMULATION ADVENTURE GAME 



Wm. J. Fetter, Paul G, Engellcirk, Janet Engelkirk, University of Texas Health Science Center 

Medical Technology Education programs typically include medical microbiology as an integral component of their 
curriculum. Many of these programs, however, neglect to teach^ or fail to emphasize the importance of anaerobic 
bacteriology. Therefore, a targe number of medical technologists lack the knowledge and experience needed to 
perform presumptive identification of anaerobic isolates. 

The Anaerobe Detective Adventure Game combines a Computer- Based Adventure Game format with an artiflcial 
intelligence knowledge base. The student interacts with the game by trying to solve a mystery using a minimum 
number of clues. The game begins at the scene of the crime. The victim is suffering from a severe infection. The 
suspect is one of 12 anaerobic bacteria, but which one? The learner, using a logical, step- wise approach must collect 
the significant clues, piece them together and solve the mystery. The game offers both an opportunity for the student 
to learn how to perform presumptive identification of anaerobic bacteria as well as sharpen their logical reasoning 
skills. 

William. J. Fetter 

University of Texas, Health Science Center 
P.O. Box 301187 
Houston TX 77230-1 187 
(713)7924466 



PEDIATRIC RESIDENT SATISFACTION WITH NATURAL LANGUAGE INTERFACE IN A 

COMPUTER SIMULATION 



Michael Foulds and Janise Richards, University of Texas Health Science Center 

Medical educators have recently become interested in computer-based interactive video instruction due to the 
deaeasing cost and increasing reliability of microcomputer hardware. Good computer-based instruction in medical 
education offers numerous benefits: active learner involvement in decision-making, opi)ortunities for multiple trials 
without harm to real-life patients, condensing time so that learner can observe the course of a disease over long 
intervals, and accessibility when ever the learner's schedule permits. Recent studies have shown that interactive video 
instruction is both effective and efficient 

A pediatric cardiopulmonary resuscitation simulation interactive videodisc has been develc^)ed to provide practice for 
pediatric residents in the development of resuscitation decision-making. In this simulation, interactive video provides 
the visual cues critical for appropriate decision-making. The situational fidelity of the simulation is enhanced by the 
use of a natural language interface. The natural language interface allows the learner to enter commands (via the 
keyboard) wriuen as familiar English phrases similar to those verbal commands given to a nurse in an Emergency 
Room. 

Resident satisfaction in using this program was evaluated using direct observations and a questionnaire. Specific 
areas addressed include comfort levels when using the interface, fidelity of the video sequences, perception of skill 
level improvement, and understand of the problem presented and approach to the problem's solution. 

D. Michael Foulds and Janise Richards 
The University of Texas Health Science 

Center at San Antonio 
7703 Floyd Curl Drive 
San Antonio, Texas 
(512)567-2280 
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STUDENT ATTITUDES TOWARDS CAI IN GROSS ANATOMY 



Anthony 7. Frisby, Ohio State University 

Student attitudes toward computer assisted instruction as a supplement or alternative to traditional laboratory 
demonstration were measured in this study. The population consisted of 72 (79%) undergraduate nursing students and 
19 (21 %) students in other allied health fields. A 27 item "Liken" scale survey was administered one week following 
the CAI and repeated (with 4 additional items) 6 weeks later to measure change over time. 

Some results from the survey arc: 14% felt that the CAI was more effective than the traditional laboratory, with 48% 
reporting preference for the lab. Even though only 34% of the students said they liked CAI. 53% reported they would 
use additional CAI programs if made available. This instrument was developed as part on the formative evaluation of 
the 'anatlab series of programs being developed by Julia F. Guy. 

Anthony J. Frisby 

The Ohio State University 

1829WiUoway Circle North 

Columbus OH 43220 

(614)292-6192 



DESIGN CONSIDERATION AND SUBJECT MATTER EXPERT INTERACTION FOR 
DEVELOPING iVD/CBT SIMULATIONS 



Tessie Hernandez and Jo Ann Crystal, JWK International, Inc, 

Conventk)nal methods for designing siuiational simulations have to be expanded when a simulation is being used to 
evalurite a studerfs decision- making skills, rather than mere recdll of knowledge. Evaluation $in:uIations must allow 
for all student selections by using natural feedback rather than artificial feedback that is used to direct students 
through a more conventional uitoriai simulation format 

In the IVD/CBT project described, four case simulations were designed for third and fourth year medical students. The 
second and third cases arc tutorial cases which provide the sttident with detailed feedback. The first and final cases are 
evaluation cases with natural feedback an^ a weighted, numerical scoring system that is used to establish student 
performance levels. 

In order to design such evaluation simulations, terminal performance objectives and detailed enabling objectives arc 
used both in the design of the case scenario and the scoring system. All scoring is uansparent to the student and the 
simulation continues naturaUy so realism and concentration are never broken. At the end of the program, students are 
given feedback rcgarding achievement of each terminal performance objective and specific enabling objectives that 
require further attention. 

Close communication between designer and subject matter experts is required to identify the performance objectives 
and detailed enabling objectives as well as the details of the case study. Throughout each phase of development and 
review, the designer must reinforce the objectives so that the final product is a true measure of the student's ability to 
achieve them. 

Tessie Hernandez 

Jo Ann Crystal 

JWK International, Inc. 

7617 LitUe River Turnpike 

Annandale, VA 22003 

(703) 750-0500 
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USING CAI AND IVD APPLICATIONS TO DESIGN NON-TRADITIONAL 
TEACHING/LEARNING STRATEGIES 
FOR USE IN A NURSING CURRICULUM 



Mary R. Price, Hunter-Bellcvue School of Nursing 

Computer-assisted insiruciion and interactive videodisc applications are used primarily to "supplement" or "enrich" 
traditional class lectures. Few nurse educators use microcomputer and video disc applications to replace traditional 
teaching strategies even though research fmdings indicate that nursing students using CAI learn more in less time 
than those receiving traditional insuiiction. The SREB's (Southern Regional Education Board) Fall 1987 survey 
indicates that less than one -tenth of the 550 respondents perceived a change in the way students learn, or a change in 
the way educators teach because of computer use in their undergraduate program. How can we anticipate a change in 
the way students learn, when in most instances CAI assignments are mere "add ons", optional assignments that they 
are not really accountable for completing. CAI assignments when required, are primarily used as adjuncts, 
supplements to the traditional classroom experience. 

The need for nurse educators to be creative in their use of computer technology is supported by the diversity of our 
current student population. Non-traditional teaching/learning strategies must be employed if we are to meet (he 
learning needs of the "non - traditional" student population entering our college nursing programs. Many are adult 
learners with diverse experiences, abilities, and learning needs. 

During the presentation, I shall outline strategies used to design and to implement Competency-Based Self-Directed 
Independent Learning Programs using computer assisted instruction and interactive videodisc applications. In 
addition, we will explore the advantages and disadvantages of a computerized learning environment and how the 
technology, when used creatively, will impact on traditional educarion. The discussion will focus on problems I 
encountered during implementation, integration tips, the educators role in the computerized learning environment, 
the student's role and their reaction to implementation of non - traditional strategies, faculty response and 
implications for curriculum design in the future. Participants will be asked to share their experiences in using 
computer technology and fo identify how the use of the technology has, or will, impact on their role, the role of 
their learners, their curriculum and course design, and administrative policy. 

Mary R. Price 

Education Consuhant 

Hunter-Bellevue School of Nursing 

Adjunct Faculty 

40 West 67th Street 

New York, New York 10023 

(212)362-9164 
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EVALUATING HEALTH SCIENCE INTERACTIVE VIDEO COURSEWARE: 

INSTRUCTIONAL CRITERIA 



James P. Randall Widener University 

A mcla-structurc for evaluating Health Science interactive video courseware (lAVC) has been proposed (Randall, 
1989). The literature indicate that demographics, documentation/support, instruction, program design and 
presentation characteristics need to be considered when evaluating interactive video courseware (DeJoy and Mills, 
1989; Foshay, et. al., 1988; Henderson and Sales, 1988). 

Therefore, this presentation describes elements which comprise the instruction component of the meta-structure for 
evaluating Health Science interactive video courseware. A review and synthesis of the literature provides a basis for 
analysis of the component parts. Interactive video program segments are used to illustrate selected elements of the 
instruction component. Discussion focuses on the need for standards for interactive video program evaluation. 

References 

DeJoy, J. K. and Mills, H. H. (1989). Criteria for Evaluating Interactive Instructional Materials for Adult Self- 
Directed Learners. Educational Technology, 29(2), 39-41. 

Foshay, R., Welsh, J., Hetzel, M., Stevens, D., and Roberts, A. (1988). The Five Generations of Interactive 
Instruction. presentation. Society for Applied Learning Technology, Summer Conference. 

Henderson, R. B. and Sales, G. C. (1988). A Guide for the Review of Interactive Videodisc Instruction. Performance 
and Instruction, 27(6), 17-21. 

Randall, J. P. (1989). Professional Standards for Health Care Sciences Interactive Video Courseware. Paper 
presentation. Society for Applied Learning Technology, Fifth Conference on Learning Technology in the Health 
Care Sciences. 

James P. Randall, Ed. D. 

Assistant Professor Instructional Technology 

Center for Education 

Widener University 

Chester, PA 19013 



DEFIBRILLATION SIMULATION 



John A. Stewart, Health Science Consortium 

Defibrillation Simulation is a tutorial/simularion program designed to help teach the proper procedure for 
defibrillation. It is written in PC/Pilot for IBM-compatible computers with EGA graphics and 640K RAM. The 
program teaches proper procedure for rapid defibrillation by providing instruction and practice in decision making and 
arriiythmia recognition. Effective integration of fundamental assessment skills and basic CPR with the defibrillation 
procedure is emphasized, and information about first-line medication is available as an option. Content conforms to 
the Advanced Cardiac Life Suppwt Guidelines of the American Heart Association. 

Defibrillation Simulation consists of two main parts: the Tutorial mode and the Simulation mode. The program 
displays simulated dynamic ECG tracings and brief text descriptions of clinical scenarios on the computer screen. 
Within the Tutorial mode the learner may determine the course of the clinical scenarios or may have the program 
generate them. The learner sees a narrative description of the correct actions to take in response to the unfolding 
events. The Simulation mode has unscored and scored options, both accepting uncucd responses which are judged by 
pattern matching. 

John A. Stewart 
Health Science Consortium 
1757 Sturgus Avenue, S. #3 
SeatUe, WA 98144 
(206) 726-0522 
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A WRINKLE IN TIME: OR USING CAD AS A TOOL IN THE DESIGN OF 
APPAREL BASED ON HISTORIC TIME PERIODS 

L_ 

Diane Davis, Valparaiso University 

The ^ppm\ industry is beginning to use CAD in the design and production of clothing. There is now a challenge to 
provide apparel design students with computer-aided design experience and to integrate this experience into existing 
courses. To meet this challenge, an elecuonic garment style library was created. It consists of garment segments 
based on styles representative of five 10-year periods, or new designs mixing components firom any of the S periods. 

Diane Davis 
Valparaiso University 
Loke Home Economics 
Valparaiso IN 46383 
(219)464-5210 



COMPUTERIZING FASHION MERCHANDISING 



Nina J. DiVoeck, California State University, Fresno 

The approach of the 21st Century brings with it a need for students to be educated in the application of new 
technologies. As with other fields, students in Fashion Merchandising need practical experiences with new 
technologies. One technology has now made it possible to teach merchandising techniques in the classroom with the 
assistance of Computer Aided Design (CAD). 

Using a customized version of AutoCAD, merchandising techniques were taught to students as a laboratory 
experience. This lab experience was an augmentation to an existing Fashion Merchandising class. One of the major 
objectives was Vj provide students, who had minimal computer skills, with the opportunity to use computerized 
merchandising techniques. The focus was on merchandising techniques rather than computer skills. 

None of the students had previous CAP program experience. With two hours of instriiction and hands-on experience 
the students were able to create diffetent nnerchandising scenarios; using the supplied floor plans and futures. Only 
simple commands such as -MOVE," * COPY,** -ROTATE," "INSERT* were needed to complete the tasks designed. 
Evaluation of an entire scenario was possible by viewing the computer screen or a printout Classroom discussions 
were stimulated by the various outcomes (hat were created for each given situation. 

Nina J. Dilbeck 

Dept of Family Studies and Home Economics 
California State University, Fresno 
Resno,CA9374(M)023 
(209)278-2283 
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THE DESIGN OF A DATABASE FOR 
AN HISTORIC COSTUME COLLECTION CATALOG 



Frances W, Mayhew, University of Delaware at Newark 

Until recently, the catalog of the Univcrsit/s Historic Costume Collection consisted of pages in a set of several 
three-ling binders. The catologing sheets were ordered by accession number. Without knowing the accession number, 
it was impossible to find a garment with a raglan sleeve, an example of Alcncon lace, a weighted silk, the boot 
panty hose or a paper dress. Using SYMANTEC Q&A and the catalog information, a database was developed 
making the Collection objects accessible to anyone who wished to use them. The Catalog Database Project included 
content analysis, development of the informational hierarchy, and inventory file design. In addition, the Project 
required the development of information collection forms, inclusion of students in the implementation process, and 
pnxluction of a User s Manual to facilitate the utilization oif the Collection resources in al) textile related programs. 

Frances W, Mayhew 
201 Alison H?il 
University of Delaware 
Newark, Debware 19716 
(302)451-8545 



CUSTOMIZING CAD SOFTWARE FOR 
APPAREL DESIGN AND PRODUCTION 



Phyllis Bell Miller, Mississippi State University 

As worldwide competition stiffens, apparel manufacturers are finding it necessary to computerize their operations. 
Apparel design and production instructors are also striving to provide students with computer skills. Because of their 
relatively low cost, microcomputer CAD programs enable small firms and schools to take advantage of quick- 
response technology. However, little customized software is available for apparel. 

AutoCAD software, which was designed for engineers and architects, was used as the basis for an apparel program. 
The program contains more than 40 new icon, screen, and tablet menus. The icon menus allow instant insertion of 
croquis figures for fashion illustration and of tools, such as French curves and L squares. Screen menus contain 
slopers, k^ls, symbols, and macros that streamline pattern making. 

Several new commands have been developed using the AutoLisp programming language. They perform such 
functions as drawing darts; calculating button/buttonhole size and placement; and adding seam allowances. On-line 
help is available for all new features. Automatic slide programs explain pattern-making procedures. The program, 
which is in use at several leading institutions and small apparel firms, takes the woric begun at the University of 
Tennessee in 1985 farther toward efficiently meeting the apparel industry*s needs. 

Phyllis Bell MiUer 
Mississippi State University 
P.O. Drawer HE 

Mississippi State University, MS 39762-5765 
(601) 325-2950 
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COMPUTER APPLICATIONS IN THE FIELD OF 
HOME ECONOMICS AND DESIGN 



Susan Winchip, Illinois State University 

Computer* Aided Design/Computer- Aided Manufacturing (CADyCAM) is an emerging technology that is now being 
used by all aspects of the design manufacturing process. A computer CADD system acquisition can become an 
integral part of a designer's work environment In an integrated computer system installation a multi-task 
woricstation is capable of executing CADD graphic functions as well as utilizing automation packages to perform 
non-graphic functions. This electronic technology has the potential to enhance the creative and technical attributes of 
design and the field of Home Economics. 

Susan Winchip 
Illinois State University 
Turner Hall 
NormallL 61761 
(309)439-2517 
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AN EXPERT SYSTEMS APPROACH TO HYPERMEDIA 



Mark ChignelL University of Southern California 

Hypermedia is a promising technology, but there are problems because it is difficult to construct and use. In this , 
Vft explore the relationship between expert systems and hypermedia^ We show how rule-based programs may assist 
in building text^ and in extracting information from hypeitext Research case studies are discussed. 

MaikChigneil 

University of Southern California 
ISE Dept. use, Gcr 240 
Los Angeles CA 90089-0193 
(213)743-2705 



USING LINKWAY IN THE DESIGN OF CONCEPT LESSONS 



Gayte V. Davidson, Deb Dorman, and Paul FayficK The University of Texas at Austin 

To date, little has been glared about LinkWay, the hypermedia authoring language for the IBM environment The 
purpose of this presentation is to describe and discuss the use of LinkWay by educators in the design of their own 
computer-based concept lessons. An example of such a concept lesson will be {vesented. 

Gaylc V. Davidson 

The University of Texas at Austin 

Dq)artment of Curriculum and Instruction 

EDB406 

Austin TX 78712 
(512)471-5211 
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DESIGNING HYPERMEDIA TRAINING MATERIALS 
TO OPTIMIZE LEARNING 



Francis A. Harvey, Lehigh University 

Lehigh University is developing an integrated hypermedia training curriculum utilizing interactive videodisc 
technology for Cities in Schools, Inc., a national organization dedicated to seeking solutions to problems of at-risk 
youth through more efficient delivery of existing social and educational services. The curriculum being developed 
provides extensive user control and user flexibility, through enhanced browsing capabilities and graphical and other 
navigational aids, in the use of a vast amount of computer-based, videodisc, and print resource materials. 

Evaluation of hypermedia materials produced during the first phase of the project indicated that, in order to optimize 
learning from the materials, users required specific guidance in selecting, sequencing, and using material. Therefore, 
hypermedia training materials now being developed include "guided tours" which present structured sequences of the 
curriculum materials organized in specific ways. The user can thus choose between unstructured exploration and more 
focussed guided learning, and can combine these approaches as needed* 

Organizational strategies used include structuring sequences based on the major themes of a training module, on 
instructional content, on learning objectives, or on the user's personal structures. This paper will describe and discuss 
how these "guided tours" and other organizational strategies were designed, developed, and evaluated, and how the 
techniques employed can be adapted for other hypermedia training applications. 

Francis A. Harvey 
Lehigh University 
College of Education 
111 Research Drive, MTC 
Bethlehem, PA 18015-4793 
(215) 758-3268 



DISAPPEARING DUCKS: PROBLEM SOLVING USING HYPERTEXT 



Beth Huning, Richardson Bay Audubon Center & Sanctuary 

The Mystery of the Disappearing Ducks is a hypermedia problem solving project developed by the Richardson Bay 
Audubon Center, Lucasfilm, and area high school students. Apple Computer provided the equipment and training. 
The result is an interactive adventure in which the scenario is set around a declining duck population, and the u<^r 
must discover why the ducks are disappearing. Through buttons you may interview experts and fanners, see ducks jp 
close or migrating, read clippings, perfomi chemistry experiments, and even interview the students who helped make 
the project happen. The program is intended for small groq) problem solving at the high school level. 
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COLLABORATIVE ANNOTATION OF A 
HYPERBOOK ON HYPERMEDIA DESIGN 



David H, Jonassen and Sherwood Wang, University of Colorado at Denver 

This presentation describes a iMocess where participants from a NATO Research Workshop on Hypermedia Design 
annotated the proceedings using qualified and unqualified annotations* conunents» and references. Annotattons were 
shared in an electronic mail conference. Implications for collaborative annotation of electronic hypertext are 
discussed 

David R Jonassen 

University of G>lorado at Denver 

Campus Box 106 

Denver, CO 802 17-3364 

(303)556-3354 



SELECTING MEDIA FOR USE IN 
EDUCATIONAL HYPERMEDIA APPLICATIONS 



David C. Klugh and Francis A, Harvey, Lehigh University 

A hypermedia-based training curriculum has been developed at Lehi^ University fwr CiUcs in Schools, Inc. (CIS), a 
national non-jront organization that coordinates the delivery of human services in a drop-out intervention program 
for at-risk youth and their families. The hypermedia training materials are being used in the training program at the 
National Center fwr Partnership Development (NCPD), a CIS training center at Lehigh University. 

This {H-esentation defines hypermedia, reviews the media selection process for non-hypermedia educational 
applications, and then discusses the adaptation of the process for hypermedia with a description of the media selection 
process used in the CIS hyp»media develq)ment project 

Typical media selection models fwr instructional plications have their basis in the models presented by Gagn£ and 
Briggs (1979) and Reiser and Gagnd (1983). The Reiser and Gagnd (1983) model queries the designer about the 
objective to be taught, following a flowchart tfiat culminates in providing dioices for media selection. The "candidate 
media** that results from this process can then be subjected to additional decision factors related to the use of 
hypermedia: the use of step-motion as compared to linear motion; whether graphics will be displayed on the 
computer-screen or on the video-screen; whether to distribute sound resources on the two audio channels of the 
videodisc or to use digitized sound; and the use of new stOTage and presentation technologies such as Compact Disc- 
Read Only Memory (CD-ROM), Digital Video Interactive (DVI), Compact Disc Interactive (CDI), and rewritable 
optical discs. 

David C. Klugh and Francis A. Harvey 
Lehigh University, College of Education 
111 Research Drive, MTC 
Bethlehem, PA 18015 
(215)738-5121 
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EXPANDING PEDAGOGICAL HORIZONS WITH A 
HYPERMEDIA CLASSROOM PRESENTATION SYSTEM 



Michael A, Kolitsky, California Lutheran University 

Our new science center contains an electronic lecture hall with a Hypermedia classroom presentation system which 
functions as a large screen electronic blackboard Video images from a videodisc player and graphic images generated 
by a Macintosh 11 computer may be displayed via a video projection monitor to produce a viewing area of 140 square 
feet. A ColorSpace II graphics overlay board is included in this system which permits the single screen display of 
not only graphics and video images but also the opportunity to mix graphics and video on the same screen and 
portends a new form of teaching in which teachers will no longer simply "write" lectures but rather will "produce" 
them. Supercard has been incorporated as the controlling software for the Hypermedia presentation system and will 
be used to demonstrate how professional vidu)graphic techniques may be employed to expand student attention span 
and explore problem solving by the production of simulation and animation sequences for large screen display. This 
new form of teaching is not only dependent on computer technology but also on the design of the classroom itself 
and a discussion of the optimal building design characteristics supportive of Hypermedia presentation systems will 
also be included. 

Michael A. Kolitsky 
Optical Data Design Center 
California Lutheran UnivCTSity 
Thousand Oaks. CA, 91360 
(805)493-3385 



HYPERTIES®: A HYPERMEDIA AUTHORING TOOL 



Charles Kreitzberg, Cognetics Corporation; and Jams Marariu, University of Maryland 

This seminar will provide a discussion and demonstration of Hyperties®, a software system for creating electronic 
books, manuals, exhibits, catalogs and other hypertextyhypermedia databases. Hyperties® is available for the IBM 
PC (XT, AT, PS/2) and compatibles, with input available using keyboard arrow keys, a mouse, or a touch screen. 
Articles may be multi-page, consisting of text, graphics, video, and audio. 

Charles Kreitzberg 

Cognetics Corporation 

55 Princeton Hightstown Road 

Princeton Junction NJ 08550 

(609)799-8555 
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INFORMATION EXPLORATION-BASED INSTRUCTION: 
GUIDELINES FOR HYPERCARD STACK DEVELOPMENT 



Annette Lamb, University of Toledo 

The words "information exploration" have been associated with HyperCard since it's introduction. The words have 
iosuirccted a heated debate in the educational community about the nature of learner control. Do sMdents flourish or 
flounder in a learner controlled environment? Is more or less guidance positive for learning? One of the keys to these 
questions regard the design of instruction. To be effective, instruction must be carefully planned. In many ways, it 
requires more planning to create a flexible learning environment than a structured learning environment The purpose 
of this presentation is to describe these guidelines and to illustrate them with examples. 

Information exploration systems can be designed to teach higho- order learning outcomes and maximize learner- 
control. However, little has been done to develop slacks that provide students with assistance wading through data 
and identifying a context for this information. If students will be expected to use this information, then the designer 
will need to provide opportunities for application, in addition to simply giving students access to the information. 
These kinds of environments would allow students to control the presentation of the events of instruction without 
eliminating important designer control over entry skills and instructional sequence. This presentation describes and 
demonstrates guidelines fw creating HyperCard information exploration-based instructional systems. Specifically, it 
proposes a model for instructional development of such instructional systems. This model includes identifying needs, 
identifying alternative methods for organizing information, identifying means of navigation, developing slack 
structure, and conducting fcMHiative evaluation. 

Annette Lamb 

The University of Toledo 

2801 West Bancroft, 2021 GH 

Toldeo, OH 43606 

(419) 537-4713 



THE WORD LEARNER 



Henryk /?. Marcinldewicz, Pennsylvania Slate University 

This is a CBI frameworic based on a course designed for second/foreign language training of industrial employees. 
The instruction comprises two parts: Word Study and Word Grouping. The intent of the instruction is for practicing 
determining the meaning and usage of unfamiliar concept$/vocabulary. 

In the Word Study screen, the concept/vocabulary item is introduced via a thought-provoking question. Several 
example contexts, etymological and syntactic clues, relational or analogical clues, and voiced pronunciation are made 
simulianeoucly accessible on one screen. The question challenges the learner. The contexts provide example usages. 
The etymological, syntactic, and relational clues provide additional information which derive from the learner's 
existing knowledge base. The pronunciation provides an auditory clue. Together these enable the learner to make an 
educated guess about the meaning and usage of the new lexical item. 

In the Word Grouping screen, the learners categorize the lexical items by idiosyncratic criteria. Relationships among 
the items must be established in order to accomplish the grouping. The grouping strengthens the retention of llie 
items. The simultaneous accessibility to these strategies is a unique application made possible by this computer- 
based rendering. 

Henryk R. Maicinkiewicz 
Pennsylvania Slate University 
257 Bradley Avenue 
Slate College PA 16801 
(814) 863-1691 x13 
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DESIGN OF HYPERTEXT: THE EFFECTS 
OF USER CONTROL AND PRACTICE 



Diane McGrath, Kansas State University 

The research to be reported was designed to assess the effect of degree of learner control of a hypertext document on 
understanding of complex material. The study used three degrees of author direction (as embodied in the hypertext 
document): (1) guided tour through several levels of complexity; (2) guide ^ tour through the entire complexity, with 
no easier levels; and (3) complete user control through the complex version of the document. High school age 
learners, one group with HyperCard experience and one group who did not have such experience until after the 
learning task, studied a social studies topic using a hypertext stack which controlled a videodisc. These groups were 
assessed for understanding by three means: a semantic network, the design of a HyperCard stack of their own to 
explicate their understanding, and application questions to assess the ability to use that knowledge (transfer). 
Although research has suggested) that poorer learners like user control better but do worse with it, while good 
learners like lesson control and yet do better with learner control, it is expected that students with practice in learning 
from hypertext will show better learning and will demonstrate the greatest transfer when they have the most learner 
control. 

Diane McGrath 
Kansas State University 
261 BluemontHall 
Manhattan KS 66506 
(913) 532-7856 



HYPERCARD DESIGN AND FIELD DOMINANCE OF THE DESIGNER 



Hilary McLellan, Kansas State University 

This paper presents the results of a study examining learning style in relationship to how students designing in 
hypercard cognitively represented the structure of hypwcard jid the structure of the information designed into the 
hypercard format. This study examined how students characterized by different learning styles linked information 
while designing hypercard stacks. Students were supplied with database information, both textual and pictorial, 
which formed the basis for designing hypercard stacks. They wo'e also provided with training in hypercard and screen 
design basics. This training emphasized viewing the user as an explorer accessing information via both guided tours 
and free exploration within the hypercard slack. Subjects were college students. The Myers-Briggs Instrument was 
used to identify learning style patterns. An assessment was made of how students arranged a representation of the 
hypercard structure and how they arranged the information presented to them for design into hypercard. These designs 
were assessed in terms of complexity, information navigation structure and strategies, information coherence and 
comprehensiveness, and overall design. The results of this study may have implications not only for hypertext 
designers, but also for learners confronted with a hypertext learning environment. 

Hilary McLellan 
Kansas State University 
226 Bluemont Hall 
Manhattan KS 66506 
(913) 532-7856 
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STARTING FROM SCRATCH: HYPERMEDIA IN THE CLASSROOM 



Louis Molinari and Lyme Levy, Glassboro State College 

The functional role the computer plays in the elementary and secondary classroom is dependent upon the computer 
expertise of the classroom teacher, the equipment available, and time allotted to planning and development of 
instructional experiences incorporating the computer. When correctly applied, the computer along with related 
technologies can produce outstanding lessons. 

When combined with hypermedia, the computer can be expanded into a tool to create dynamic and worthwhile 
learning environments. With some effort and lots of time, the classroom teacher can create, design, and organize 
quality interactive educational lessons utilizing hypermedia software which can put the computer and laserdisc player 
or VCR together as one highly effective delivery system, 

A hypermedia environment can be created with a minimum of additional spending, utilizing the Apple II series of 
computers, along with a laserdisc player and interface, authoring software and videodiscs, and possibly a video 
overlay card. The teacher need not be a programmer since most hypermedia software provides the teacher with step- 
by-step instructions for lesson preparation. 

Hypermedia programs for the Apple II computers which utilize a videodisc and/or a VCR to provide interactive 
video, include HyperStudio, Tutor Tech, VCR Companion, and LaserWorks. With the union of hypermedia 
technologies, not only does the teacher have the opportunity to present effective lessons, but the teacher can create 
and control dynamic learning environments. 

Louis Molinari and Lyime Levy 
Glassboro State College 
Glassboro, New Jersey 08028 
(609) 863-7042 



INTERACTION INSTRUCTION DEVELOPMENT USING PC TOOLBOX 



Claude Ostyn, Asymetrix Corporation 

A powerful new software tool for the creation of interactive hypermedia applications on IBM and compatible PCs is 
now available.PC ToolBox is also ideally suited to instructional applications and hypertext. The session will 
demonstrate how to the features of PC ToolBox can be put to work in the development of various kind of 
instructional materials. 

Claude Ostyn 

Asymetrix Corporation 

110 UOth Avenue NE, Suite 717 

BeUevue WA 98004 

(206)462-0501 
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THE HYPERCARD AND QUEST AUTHORING ENVIRONMENTS: 
TWO APPROACHES TO CBI 
COURSEWARE DEVELOPMENT FOR THE "NOVICE'' 



Steve Purcell 

As education and industry have shown greater interest in using CBI courseware, newer authoring systems have been 
marketed to assist instructional designers in developing these programs. This presentation, through comparisons and 
contrasts, will examine one MS-DOS and one Macintosh-based authoring system (QUEST and HyperCard, 
respectively), both of which w»e designed with the **novice** CBI couisewa^e developer in mind. 

Steve Purcell 

501 TSunridge Drive 

BlacksburgVA 24060 

(703)231-5587 



THE EVOLUTION OF A HYPERCARD STACK: 
NEGOTIATING SPANISH WEEDS 



Lawrence C. Ragan, Pennsylvania State University 

The develcqHnent of courseware using HyperCard presents several unique and challenging management problems to 
the development team. Due to the flexible nature and the internal structure of HyperCard (i.e., stacks, cards, fields, 
backgrounds, buttons and scripting), revisions and modiflcations are more easily instituted in the courseware. These 
modifications can enhance the jmiuct or add to the development time and costs. This presentation will examine 
these issues through a look at the evolutionar>' stages of courseware development as well as the factors that might 
influence these stages. 

The Computer Based Education Lab-Teaching and Learning Technologies Group at Penn State is a faculty suppoit 
unit that develops courseware for faculty through the team model consisting of content experts, instructional 
designers, instructional programmers, and graphic artist Through experience, the instructional design teams at the 
CBEL have learned much about HyperCard as a courseware development environment CBEL has been utilizing 
HyperCard since the introduction of the program in 1987. Courseware developed at Penn State will serve as the 
model in discussing the development process and the **evolution** of an educational HyperCard stack. Currently five 
products in HyperCard are under development ot in testing. 

This presentation will include the following: 

• An Introduction to The Penn State Courseware Development Process 

• HyperCard as an Instructional Solution: Some Pros and Cons 

• The Evolution of a IJ- pciCard Stack 

• Summary 

Handouts and demo versions of HyperCard courseware will be available for interested participants as well as a list of 
the resources used in the development process. 

Lawrc^ce C. Ragan 
Instn uonal Designer 
Penn State University 
12WillardBldg. 
University Park, PA. 16802 
(814-863-1654) 
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A PODIUM FOR TEACHING PHILOSOPHY 



Donald Scherer, Bowling Green State University 

Like professors of history, law, literature and theology, professors of philosophy teach students how to manipulate 
texts. Successful students in each of these fields learn how to recognize symbolic patterns: ambiguities, definitions, 
assumptions and conclusions, for instance. PODIUM, devel A at the University of Delaware, is adaptable to 
facilitate classroom demonstration of these manipulations. 

The pedagogical problem is that professors typically want to use students' recognition of symbolic patterns as a 
means to helping students to understand the content of their discipline, but students enter courses with no better than 
a low awareness that the patterns exist. Thus the professors are constrained to teach symbolic pattern recognition, 
even though it is often boring for students and even though learning it often takes time away from the course 
content. 

A sophisticated use of PODIUM allows PODIUM itself to code different patterns (e.g., term to be defined, terms in 
which it is defined) so that the coding familiarizes students with the symbolic patterns, recnforces instruction 
concerning the patterns and demonstrates to students throughout the semester the usefulness of recognizing the 
pattern. The paper details the potential of such coding for acquainting students with the systemic relations of the 
symbolic patterns employed in a course. 

Donald Scherer 
Philosc^hy Dq)artment 
Bowling Green State University 
Bowling Green, OH 43403 
(419) 372-7142 



HYPERMEDIA, CRITICAL THINKING, AND DIAGNOSTIC SIMULATIONS IN 
THE HEALTH SCIENCES: A DEVELOPMENT MODEL 



Eiwin /?. Tilson, Armstrong State College 

This papci ouUines a game-like simulation development model uses hypermedia, gaming theory, diagnostic 
modeling, critical thinking theory, and modified guided design looping. Computer simulations are far from a new 
idea in training and education. Unfortunately, simulations were historically either very elaborate, expensive, and time 
consuming or were not very realistic. Recent advances in technology such as hyper-text, CD ROM, scanned images, 
digitized sound, animation, and non-language based authoring systems have allowed for development of substantially 
improved micro-based simulations. 

Examples from an Apple Academic Development Grant are used to demonstrate the implementation of the model. In 
the project, business students participate in "A Day in the Life of a Manager" simulation. All decision points in the 
simulation require the use of a specific mathematical analysis of business data to determine the appropriate action. 
Also within the gaming simulation, profits and losses for the day are used as gaming tools to keep the learner's 
interest high. When a performance deficiency is diagnosed, the learner is moved from to a modified guided design 
sequence that gives remedial instruction and models critical thinking approaches to specific information analysis. 
Upon completion of the guided design sequence, the learner is placed back into the simulation. 

Elwin R. Tilson 
Armstrong State College 
Health Science 
Savannah GA 31419 
(912) 927-5360 
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PLATFORM INDbPENDENT AUTHORING AND DELIVERY OF 
INTERACTIVE MULTIMEDIA 



Michael W, Allen, Authorware 

The new CBI is alive with integrated sound, animation, video, color, graphics and text Much is being achieved at 
surprisingly practical costs and developed nq>idly with icon-based authoring systems. Unfortunately, most 
applications are tied to very speciHc hardware conflgurations. This presentation will demonstrate nearly identical full 
multimedia audKMing and delivery on both Macintosh and Windows platforms with exchangeable courseware files. 

MichadW.AUen 
Authorware 

8500 Nonnandak Lake Blvd, 9th Floor 
Minneapolis MN SS437 
(612) 921-9555 



YOU'VE GOT A FRIEIID IN THE INTERACTIVE BUSINESS: 
PROGRAMMING THE AMIGA FOR INTERACTIVE VIDEO 



J. Wesley Baken Cedarville College 

This session introduces the Amiga as a platfmn (cx interactive video and presents the progranuning language and 
authoring programs available. An interactive video program being used at Cedarville Q)ll<'ge to introduce faculty and 
administrators to the technology will be demonstrated. This program resulted in the funding of an interactive design 
team which began work in the summer of 1990. 

To provide a comparison between the authoring options available for the Amiga, three separate versfons of the 
program will be shown at the session. One is written in Amiga PILOT, the other two arc written using icon-based 
authoring programs which take different i^jproacbes - VIVA and AmigaVision. 

While Amiga PILOT maintains its su-ength as a specialized language for interactive programming it is not widely 
available and requires writing code. VIVA was the first interactive authoring program for the Amiga to use icons, but 
its structure is somewhat confusing. AmigaVision provides a flowchart-like structure, making it convenient to 
quickly create programs. Although neither of the icon-based authoring programs completely eliminates the need to 
understand programming logic, they do make the creation of interactive programs more accessible to those 
apprehensive about writing lines of code. 

J. Wesley Baker 

Department of Communication Arts 
Cedarville College 
P.O. Box 601 
Cedarville, OH 45314 
(513)766-2211x489 
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TRANSLATING AND REPURPOSING VIDEODISCS: 
THE DEVELOPMENTAL PROCESS IN TAIWAN 



Jiin-Tian Chyou . Tamkang University: and Bor-Nian Chang, ISTEC 

This prcsenlation will demonstrate an application of an interactive video program using Chinese translation and 
instructional design strategi ts to promote efficient learning. Software tools were developed and used to create Chinese 
captions. This tool can storage and generate Chinese character, and create text files by text editor. It helps the 
developer translate English videodisc materials on screen into Chinese texts to promote IVD development 

The structure of Chinese fonts and the development process of the program tools to facilitate translation will be 
discussed. The design process of repurposing "Advanced Interactive Technologies Disc" for ISTEC (Information 
Science & Technology Exhibition Center) in Taiwan will be presented. Design elements include level of information 
provided, menu variation, moving icons, on-line help animation, voice feedback, etc. These strategies is incorporated 
to enhance efficient interaction. The hardware and software tools used in developing the program include two PC/AT, 
the Chinese translation too], graphic program, video overlay card, and a videodisc player. 

The potential develc^ment of multimedia programs and future work will be presented. These include expanding 
design options for repurposing videodisc materials, data collection to support evaluation, and planning multimedia 
learning lab to promote IVD applications. 

Jiin-Tian Chyou 

Educational Media & Library Sciences 

Tamkang University 

Tamsui, TAIWAN 

(02)621-5656 Ext 309 

Biuiei: TKUT041@TWNMOE10 
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THE APPLICATION OF FORMAL DESIGN THEORIES TO THE PRODUCTION OF 
VIDEO/CBI INSTRUCTIONAL SYSTEMS 



Charles R. Dills and Alexander Romiszowski, Syracuse University 

Abstract 

Several tomal instructional design models are reviewed, and 

their 2^plicability to the design of CBI/IV instruction based upon videodiscs and digitized graphics is discussed. 
Characteristics of the next generation of design models are specified in terms of what is needed to overcome proU^s 
currently faced in the design of such lessons using formal models. A second-generation model currendy under 
development is presented. 

Summary 

Several formal instructional design models have received extensive develq)ment and verificational evaluation over the 
past fifteen years (A large sample of these are described in Charles Reigeluth, Instructional Design Models). Also, 
there have been developed a good many formalized uKxIels of leaching(many of these are descrioed in Joyce and Weil, 
Models of Teaching). The teaching models have been primarily developed for use with specialized purposes, while 
the design models have been rather more broadly based 

Reigeluth has argued that task analysis and instructional design should be integrated, and has produced his Extended 
Task Analysis as an example of how this might oe done. The authors argue that a further integration is necessary; 
namely, that teaching models, design models, task analysis and media selection all should be treated as an integrated 
procedure. This presentation presents arguments to that effect, discusses current teaching models and design models 
in terms of how they relate to each >lher, and describe a meta-model for achieving the proposed integration. A 
computer program for selecting an appropriate teaching model and a design model for given objectives is 
demonstrated. Its extension to media selection is described, and plans for verification studies involving a variety of 
audiences and settings are ouuined. 

Charles R. Dills 
Syracuse Univwsity 
330 Huntington Hall 
College of Education - roD&E 
Syracuse NY 13244-2340 
(315)443-3703 
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STEPS FOR DEVELOPING AWARD WINNING 
INTERACTIVE VIDEO SCRIPTS 



Deidre Emery and Steven Holder, Southwestern Cable 

A successful interactive video begins with a comprehensive pre-production development strategy. This strategy 
anticipates design problems and corrects them immediately, thereby decreasing production costs and minimizing 
manpower requirements. Designing an interactive video is complicated and mentally taxing. An instructional writer 
must develop an interesting and complex script that entices the viewer to learn while staying within tight budget and 
time constraints. 

The instructional writer becomes a concept juggler. S/hc is responsible for script content, dialog, learner feedback 
loops, set design, talent directions, special audio/vicual effects, as well as, shooting and editing needs. S^e decides 
wh£t information is to be presented in the form of graphics, voice overs, "live" video, special effects and still 
pictures. Fmally, §/he times each learning segment to insure flawless intwaction among the laser disc, CD-ROM and 
desk top computer. 

A systematk: method for information researching, brain storming, flowcharting, labeling, writing and sorting is vital 
to the instructional writer's project success and personal sanity. IVID Communications, a recent Cindy Award 
winner, prepares its writers for the comidicated task of designing interactive video scripts by providing them with a 
proven successful pre-production develc^ment strategy. The strategy includes: Developing master I VD questions for 
learner achievement; Fk)wchart feedback (branching) designs; Writing edits and pitfalls; Graphic and screen design 
options; Special Effects (audio and video); and Computerized scene sorting according to shooting and editing 
prerequisites 

Deidre Emery 
Southwestem Cable 
5720 Marengo Avenue 
La Mesa C A 92042 
(619)461-8434 



HEALTH PHYSICS TECHNICIAN COMPUTER-BASED TRAINING 



Kay Houghton and Lois Spangenberg, Los Alamos National Laboratory 

Health Physics Technicians provide a critical function in scientific laboratories. They ensure safe operation of 
facilities, respond when problems arise, and monitor daily experiments. This task requires health physics 
fundamentals, and knowledge of specific procedures used in responding to incidents. At Los Alamos National 
Laboratory we are develc^ing an eight module computer-based training course to train new technicians and refresh the 
skills of current technicians. 

Part of the technician's job is to select and use complex monitoring instruments. In this showcase demonstration, 
the computer-based training nKxiule on instrumentation will be shown. The module allows the employee to select 
the correct instrument for a given assignment, perform an operational check-out of the instrument, read the 
instrument's meter, and accurately interpret the given meter reading. 

Notable features of this computer-based training program include: A high degree of learner control through an iconic 
command bar; Flexibility for different learner needs; Scanned color images with graphic overlays; Storage on CD- 
ROM; Simulated task performance; High quality and frequent user interaction; High graphics to text ratio; Digital 
audio (both voice and sound effects); and Feed back tailored to the particulai user response. 

Kay Houghton 

Los Alamos National Laboratory 
MS M997, Group A-6 
Los Alamos NM 87544 
(505)667-8010 
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IVD/CAI FOR THE TEACHING OF INTERMEDIATE LEVEL LANGUAGE 



Anita M. Knisbacher, US Department of Defense 

The use of interactive videodisc is taken to new levels of sophistication in a Hebrew language project designed and 
developed by the author as part of a 5 year I>oD research effort Based on the best of Israeli TV, the two courses, 
prototype and final product, employ a variety of contextual exercises— CLOZE, scrambled sentences, word chaining, 
sequencing, gisting, etc.— to teach all facets of the language including the four basic skills of listening, speaking, 
reading, and writing. Also included is an in-depth review of gramnior and detailed excursions into idiomatic 
expressions, word formation, and the subtleties of Israeli culture. 

The educational design is equally innovative and thorough. At every step, the student has the option of taking 
diagnostic tests to bypass virtually any part of the course. Indeed, freedom of movement through the content is 
ui^xualleled and is only limited when the student has elected to take a test 

Efficient use of time and material is encouraged by a multi-level access approach thai advises the student to delve 
into greater detail — to drop down from exercises at the disc level to lesson level to segment i.;vel — according to 
performance on the various drills, exercises, and tests. 

Anit?. M. Knisbacher 
US Dqxirtn^i: Defense 
12331 Tlmbeig^o Road 
Owings MUls, MA 21117 
(301)859-6901 



DESIGN, PRODUCTION AND USE OF 
A VERTEBRATE EMBRYOLOGY VIDEODISC 



Michael A. Kolitsky, California Lutheran University 

A Hypermedia Laboratory was configured containing fourteen Macintosh n computers each interfaced to a videodisc 
player. The first videodisc produced by the University was utilized in the HypCTmedia Lab to simulate microscopic 
study in a Vertebrate Embryology laboratory course. The embryology videodisc contains images of all the 
microscope slide material normally viewed in a traditional Vertebrate Embryology laboratory course. The videodisc 
was designed to simulate focusing and utilized computer graphics in a highly integrated manner to better assist 
students in their task of visualizing a three dimensional construct of the embryo from the two dimensional serial 
sections. The interactive videodisc learning stations in the Hypermedia Laboratory were able to substitute for 
microscopes and the embryology videodisc served as an easily accessible slide bank and graphics atlas for the students 
enrolled in the course. Student response was strongly in favor of continuing this mode of instruction in future 
embryology courses and recommended its inclusion in other microscope-based laboratories as well. Videodisc usage 
will be demonstrated and the undcriying pedagogy discussed. How to design and produce a videodisc on a shoestring 
budget will also be covered and the institutional infrastructure necessary for a campus-wide Hypermedia effort will 
also be included. 

Michael A. Kolitsky 
Optical Data Design Center 
California Lutheran University 
Thousand Oaks, CA 91360 
(805)493-3385 
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DEVELOPING MULTIMEDIA TRAINING TOOLS 



Helen B. Miller and Andy McGuire, Clemson University 

The presentation will focus on the challenge of developing a multimedia training tool that utilized procedures dictated 
by the systems ^roach. Two specific issues to be addressed are the importance of team building in developing a 
creative, productive development team and living within the constraints of the chosen software and hardware 
configuration. 

The background for this presentation was a grant presented to Clemson University requiring the design of a stand- 
alone training tool that would [M'omote technology transfer to industries in the Appalachian States. The training 
program uses state-of-the-art technology, such as digitized video and audio, touch-screen input, and animatics. 

The food service industry was targeted because this industry has been plagued with high employee turnover, low job 
satisfaction, limited resources for training, poor employee-managementAnanagement-employee communication skills 
and low educational levels of nK)st employees in line/staff positions. Using a multimedia training tool, individuals 
progress at their own pace while learning to deal with real world sanitation issues through computer simulations. 

To design an effective educational program for training in the food service industry, a multi-disciplinary team 
^>proach was utilized. Instructional designers, video q)ecialists and programmers worked cooperatively with content 
q)ecialists to decide target audience, instructional strategies, and the skill competencies required by the food service 
employees. 

Helen B. Miller 
Clemson Univereity 
1101 ShOTeviewDr. 
oeneca, SC 29678 
(803)656-0626 



VIDEODISC-BASED HEALTH PROMOTION: 
EDUCATION PLUS MOTIVATION IN SMOKING CESSATION 



John Noell, Anthony Biglan, and Don Hood, Oregon Research Institute 

The Smcdcing Cessation Touch-Screen System (SCTSS) was developed as part of a program project entitled 
"Tobacco Reduction and Cancer Control** (TRACQ, funded by the National Cancer Institute. The system is a 
prototype, based on an clinically-proven smoking cessation intervention that used a health educator and a short 
videotape on smoking cessation. The SCTSS system rq)licates the activities of the health educator and integrates the 
activities of the health educator as well as the videotaped material into an interactive format 

Patients who smoke are asked to use the system in the physician's office. It amplifies the physician's brief advice to 
quit and provides information on the advantages of stopping smoking. Using an approach designed to reduce 
resistance to the idea of quitting cigarettes, it includes motivation as an integral part of education. 

The system uses a Macintosh (such as the SE20 model) to control a videodisc player in response to the detection of 
touches on monitor equipped with a touch-screen. Only the monitor is exposed to view. The touches on the screen 
replicate mouse "clicks" and are mapped to the Macintosh screen where they are interpreted as "clicks" in a 
HyperCard-based program. 

John Noell 

Oregon Research Institute 
1899 Wilamette 
Eugene OR 97401 
(503)484-2123 
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ENSURING INSTRUCTIONAL SOUNDNESS 
THROUGH SOUND EVALUTION 



William W. Lee and Kenneth H. Roadman 

Interactive Videodisc training, as with any type of training, will only come into its own when its effectiveness for 
improving performance in the workplace is documented through carefully planned and implemented evaluation 
activities. This presentation suggests a model for thorough, fool-proof, evaluation to enter each phase of training 
course production and the rationale for each activity. 

For the purpose of clarity, we have separated the evaluation process into six distinct phases. These phases may not 
be as cleariy differentiated in the production process used by individual companies but the intent is that all activities 
should be considered in order to tailor the final training product to the needs of the client These six phases are: 
Marketing, Program Planning, Analysis, Design, Development, and Implementation. 

During the Marketing Phase, the client's expectations fw evaluation are established. During Program Planning, 
client expectations are finalized so a cost base for including evaluation in the overall product development can be 
established. The Analysis Phase is wh^ needs assessment and firont-end analysis are conducted Design involves the 
establishment of the formative evaluation plan, validating the content, and preliminary work for summative 
evaluation. Evaluation during the Development Phase involves completion of the formative evaluation components 
in courseware production. The Implementation Phase involves the completion of the formative evaluation plan and 
conducting a summative evaluation. This presentation shows that evaluation is not something to be tacked on after a 
course has been developed. L must be woven throughout the entire production process. 

William W.Lee 
Educational Technology 
351 Wingate Court 
Millersville PA 17551 
(717) 872-2615 



AN INSTRUCTIONAL DESIGN MODEL FOR THE DESIGN AND DEVELOPMENT OF 

INTERACTIVE VIDEODISC PROJECTS 



William D. Milheim and Harold J. Bailey, Bloomsburg University 

While a number of well-known instructional design models exist for the development of computer-based materials, 
few specifically address the issues relevant to the design and production of instructional, interactive video programs. 
With the lack of information in this area, the Institute for Interactive Technologies at Bloomsburg University 
developed a design model specifically focused on the development of interactive video lessons. 

This presentation will include an overall description of the model, its theory bases, and its application in interactive 
video projects. Specific topics will include: 1) the overall coordination of a project, 2) design team responsibilities. 
3) working with subject matter experts, 4) use of project timelines and milestones, 5) use of content outlines, 6) 
flow-crianing, 7) scripting, 8) guidelines for screen design and menu usage, 9) documentation, and 10)project 
evaluation. The presentation also includes a number of examples which demonstrate the «nodcrs use in various 
settings in education and industry. 

William D. Milheim 
1210 McCormick Center 
Bloomsburg University 
Bloomsburg, PA 17815 
(717) 3894849 
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A VIDEODISC FOR THE TEACHING OF 
FRENCH LANGUAGE AND CULTURE 



Donna Mydlarsid, The University of Calgary 

Developers at the Universities of Calgaiy and Guelph have designed an interactive videodisc to teach French as a 
second language firmly rooted in a French Canadian cultural setting. It was decided to use top quality video from the 
flhn library of the Canadian Broadcasting Corporation and add to a fifteen-minute film a variety of slides and 
drawings. A wide range of audio and computer-based materials have also been added. 

VI-CONTE means a **video-story** and uses narration for teaching aural comprehension with cultural materials derived 
from and supporting the animated short Him which is at the core cf tite program. The basic principle is the 
coordinate presentation of a sentence with a clarifying situation. 

The disc begins with an iconic situation, a Him without words (although the music and sound effects are very 
expressive). The situation being understood, the program goes on to supply concrue vocabulary. Then, once wo'd 
has become attached to image, simple sentences in the form of questions reinforce the isolated words and begin 
setting them in verbal context There is no writing on the screen. Iliree or four verbal choices are spoken, with only 
wriuen letters appearing on the screen. The student types in the letter corresponding to the choice. Feedback is 
completdy oral 

The first poccss is called "observation". It is followed by a process called "narration**, whose overall linguisuc goal 
is to create a grammatical firamework for the data learned in **observation**. First, a verbal story line is attached to the 
visual story. Written questions are asked about the story as well as about inferences the student may make from the 
visuals. Then the student hears three synopses of the votal story line which are exploited via cloze, multiple chok:e 
and dictation with written feedback. 

The final process is called **reflexion'*. An illustration of the traditional French Canadian legend. La Chasse Galerie, 
is presented on the screen. The student has a choice of two sound tracks to practice oral comprehension. The first is 
in standard CBC Canadian French; the second in colloquial Quebec speech, with a regional accent. Finally, there are 
photos of the Quebec City carnival, followed by art slides and many photos of modem day Montreal and Quebec 
City. This is a multi-level disc directed at several different learning populations. There is no expectation that learners 
will pass from **observatk)n** to **reflexion** by using only the disc. 

Note: The support of the following is gratefully acknowledged: Canadian Studies Directorate, Department of the 
Secretary of Slate (Canada); Canadian Broadcasting Cbrporation; The University of Calgary, University Radio and 
Television Committee and Special Projects Fund of the Board of Governors. Cest aux admirables images de Frederic 
Back que ce projet pedagogique doit son inspiration et sa realisation. 

Donna Mydlarski 
The University of Calgary 
Faatlty of Continuing Education 
CrJfeary Alberta T2N 1N4 
(403) 220-5009 
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EFFECTS OF VARIATIONS IN LESSON CONTROL ON CHILDREN'S CURIOSITY AND 
LEARNING ABOUT ART FROM INTERACTIVE VIDEO 



Marilyn Johnstone Plavocos, Syracuse University 

Encouraging curiosity in young learners is an important task for educators yet, in the area of interactive video, 
literature related to this topic is lacking in empiricsd research. Most research sUidies involving interactive video and 
learns control have been conducted with sophisticated learners (Balson, Manning, Ebner & Brooks, 1985; Gay, 
1986; Hannafin & Colamalo, 1987; Gay & Trumbell, 1988). 

Issues related to lesson control can have different implications for children (Hannafin, 1984). This study investigated 
the comparative effectiveness of variations in lesson control on children's curiosity and learning from interactive 
video and whether interactive video can be utilized to both stimulate the important scholarly attribute of curiosity 
and provide adequate direction and guidance for young learners. 

The study used a post -test-only control group design with three experimental treatments. The lesson content 
provided by a subject matter expert in art education, was designed with no text to accomnKxlate subjects with limited 
reading experience. The subjects wCTe one hundred first and second grade suburban elementary school students who 
were randomly assigned to (»ie of the four study groins. 

The Designer-CcMitrolled Group followed a linear path through the lesson, the Learner-Control Group had control of 
content, sequencing, pace, and remedial needs, the Leamer-CcMitrol with Advisement Group was given the same 
opportunities to explore the lesson as the learner-control group but "advisement** strategies were employed. The 
Control Group received only the post test Study results will be part of the ADCIS presentation. 

Marilyn Johnstone Plavocos 
2030 Erie Blvd. East 
Syracuse, New York 13224 



RESULTS OF THE FIELD TEST EVALUATION OF AN INTERACTIVE VIDEODISC 
SCIENCE CURRICULUM WITH A FOCUS ON STUDENT 
AND TEACHER PERCEPTIONS AND IMPLEMENTATION 



Wilhelmina C. Savenye, The University of Texas at Austin 

This presentation will describe the field test of the Texas Learning Technology Group's full year videodisc and 
computer-based high school physical science curriculum. The test was conducted in 37 school districts in four states. 
The primary focus will be on student and teacher attitudes toward the IVD curriculum, and how it is actually being 
used in the classroom. 

Wilhelmina C. Savenye 

The University of Texas at Austin 

Dq)artment of Curriculum and Instruction 

EDB406 

AusUnTX 78712-1294 
(512)471-5211 
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TEACHING INTbRACTIVE VIDEO DESIGN AND DEVELOPMENT: 
METHODS AND MODELS 



Wilhelmina C, Savenye» The University of Texas at Austin 

Interactive video development is increasingly being taught in Instructional Technology programs. This presentation 
will describe a two-course sequence developed at the University of Texas at Austin. The curricula and goals, course 
procedures and projects, models for teaching, methods to encourage and support interactive video teaching and 
research projects, and recommendations and resources for teaching interactive video will be presented. 

Wilhdmina C. Savenye 

The University of Texas at Austin 

DqMfftment of Curriculum and Instruction 

EDB406 

Austin TX 78712-1294 
(512)471-5211 



INDIVIDUALIZING INTERACTIVE VIDEODISC INSTRUCTION 



Karen Tichy, Illinois Institute cf Technology 

The visual design field uses visual representation such a diagraming, mapping and visual models to aid in the 
analysis of design problems. This designer has iq>plied design methodology to IVD design. Using these methods, a 
model of learning was established. The model consists of the 1) user (or learner); 2) the learning ^system (be it 
traditional classroom or computer based); and 3) information or content. Each of these components were further 
analyzed and characterized Forinstance: 

A component of the learning system is the media or technology used to present information. Since IVD is the media 
being used in this case, further analysis of that media was done. This analysis identified characteristics of IVD that 
change the way peqde access and understand information* It increases the choices of available sign systems (forms of 
information such as words numbers, pictures, and diagrams). It increases the variety of activities, physical and 
mental, through which a user can interact with information. And, it makes it possible foe the user to control both 
the sign systems and the activity through which infonnation is accessed and understood 

These characteristics make it possible for the user to adapt his learning experience according to his/h^ int^ests and 
cognitive prefeiences and educational goals. 

This thesis proposes a usi t controlled learning system capable of presenting information according to user affective 
and cognitive needs and riesires. The system put; the user in control of the presentation style. The presentation style 
refers to the esthetic sense and method of presentation (straightforward lecture, or entertaining dramatization). It puts 
the user in control of the mode ot sign system used to present the infonnation and it coaches the student to mak'^ 
effective decisions about how to au:ess, manipulate^ and think about infonnation. 

Karen Tichy 

Institute of Design 

Illinois Institute of Technology 

3360 South Sta*^ Street 

Chicago, IL 60616 

312.567.3250 
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EVALUATION OF A LABOR AND DELIVERY VIDEODISC SIMULATION 



Elizabeth E. Weiner, Barbara Gilman, and Jeffry Gordon, University of Cincinnati 

The purpose of this evaluation project was to determine the impact of an interactive videodisc (IVD) simulation in 
labor and delivery on clinical confidence and learning with junior level nursing students. 

Research questions addressed were: 

Is there a difference in clinical confidence and learning between students completing an IVD simulation in 
combination with a clinical experience versus those students completing only a clinical experience? 
Does the order in which the students receive the IVD program affect the clinical confidence and learning 
outcomes? 

• Following completion of the IVD program, what overall evaluation comments do students have? 

Two clinical groups of students were created. Both groups were pretested on knowledge and clinical confidence. 
Group 1 completed the IVD program before providing clinical care in the labor setting. Group 2 received a traditional 
clinical experience first, followed by the IVD program. Multivariate am; ysis of variance supported the following 
conclusions: 

Students completing an IVD program i lus the clinical experience had sigiiificantly greater clinical confidence and 
learning than those students completing only the clinical experience. The order of the IVD program made no 
significant difference in clinical confidence or learning, but the IVD appeared to contribute more to clinical 
confidence than did the clinical experience. 

Elizabeth E. Weiner 
University of Cincinnati 
College of Nursing and Health 
CincinnaU OH 45221-0038 
(513) 558-0183 



STUDENTS AS PRODUCERS: A MULTI-MEDIA PROJECT FOR MIDDLE SCHOOL 
MATHEMATICS, SCIENCE, AND SOCIAL STUDIES 



Michael Young, University of Connecticut; and 
William Corbin and Susan Williams, Vanderbilt University 

An investigation of the topic of students as producers has been proposed. The project, supported by BM. McDonnell 
Foundation, and Vanderbilt University would take students from the position of learners to being producers of 
knowledge in 4 steps. First, students become familiar with multi-media technology. Second, students receive 
instruction in mathematics, science, and social studies using the multi-media formats they will later produce, 
particularly the "Jasper Series". Third, students choose, plan, and research topics introduced through instruction using 
conventional and electronic resources (including fax, e-mail, and databases). Finally, students create presentations of 
their findings using a combination of videotapes, audiotapes, videodiscs, and hypermedia programs. 

Michael Young, Vanderbilt University 
Learning Technology Center 
Box 45 Peabody College 
Nashville TN 37203 
(615) 322-8070 
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AN EXPERIENTIAL FRAMEWORK FOR EVALUATING SIMULATIONS 



Wayne Anderson^ JWK International 

Simulations are often judged as "successful" based on their degree of acceptance by the learners involved. Yet 
simulations are often employed in projects with no criteria for evaluating experiential components such as 
identification and participation. I>ecisions to use simulation in training often are dictated by client desires or budget 
constraints rather than their effectiveness for the learning process. How does one structure design in the beginning to 
promote learner accq)tance of simulation? How does one measure experiential components of a simulation? This 
presentation suggests a framework of three interpersonal variables for evaluating the effectiveness of simulations and 
implications for future design. Methods for achieving learner acceptance must consider what helps the learner 
"belong" to the experience. Simulations should mimic the ways in which people process information as well as 
solicit and give feedback. These methods must be integrated into the design process before "development" begins. 

Wayne Anderson 
JWK International 
7617 LitUe River Turnpike 
Annandale. V A 22003 
(703) 7500500 



TEAMWORK: GROUP DEVELOPMENT OF CBT 



Kay Bonham and Karen Dron is, Goal Systems International 

We will begin the presentation with a group discussion of the benefits of using a team for developing computer- 
based training. >Ve will ask participants to describe anecdotes of their experience with the use of teams in CBT 
development. Building on these contributions, we will generate a list of the advantages and disadvantages of group 
development of CBT. 

Next, we will present several principles of group communication. We will show how effective use of these 
principles will enhanc^ 'he advantages of group development while minimizing the disadvantages. To illustrate these 
principles, we will .onduct several exercises with the participants. These exercues will show how group 
communication and group structure affect group performance. 

At this point, the participants will split up into several development teams. Each development team will be given 
the same project assignment We will give each participant a profile of the rile he or show will play within their 
development teams. This profile will include a list of each participant's personal goals. The teams will have the 
same project assignments, but different participant profiles. We will then ask each team to develop their team goals. 
We will show how persona) goals of individuals affect the goals of the team and how a project manager can 
recognize these hidden agendas and minimize their affect on the group goals. 

Kay Bonham 

Goal Systems International 
7965 N. High Street 
Columbus, OH 43235 
(614)888-1775 
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OUTGROWING THE COTTAGE: 
DEVELOPING COMPUTER-BASED TRAINING ON A LARGE SCALE 



Robert C. Fratini. AT&T National Product Training Center 

In 1986, AT&T Product Training Services be5an the development of its first large-scale computer-based training 
curriculum, to support maintenance training for its flagship central offlce product, the SESS Switch(R). At this 
time, CBT develc^ent expertise resided primarily in a handful of CBT developers who lacked detailed product 
expertise. Tools and processes evolved to facilitate the hand-off of content from content specialists to CBT 
development specialists. 

By 1988, we had become victims of our own success. The CBT curriculum had been successfully implemented 
woiidwide, but the co<:t5 of maintaining and adding to the curriculum were perceived to be unacceptable, especially in 
terms of the life cycle of the SESS Switch product itself. 

In 1989, the Electronic Training Solutions (ETS) department of PTS began the development of a strategy to 
facilitate the large- scale development and maintenance of computer-based curricula. This involved an analysis of the 
PTS organizational, political, and training funding structures. Processes were then developed taking these structures 
into account, and CBT development tools were selected or developed to support the processes. The goals of these 
changes were to minimize the need for the develq)ment of interim products and product hand-of fs, while maintaining 
the presence of CBT specialists in those areas deemed by PTS management to be most appropriate. 

Robert C. Fratini 

AT&T National Product Training Center 
3151 Blazer Memorial Parkway 
DubUn, Ohio 43017 
614-764-5441 



DEVELOPMENT AND IMPLEMENTATION OF A COMPUTER-BASED DATA 
MANAGEMENT SYSTEM FOR NOVICE COMPUTER USERS 



Francis Hensley, University cf Georgia; and Chris Peters, Clemson University 

This presentation will demonstrate and describe the development of a computer-based management system for 
members of a national netwoik of educational project managers. This application was developed using a sophisticated 
data base management system. Fourth Dimension, especially for use by novice computer users. The system exploits 
the visual interface and ease of use of the Macintosh computer and allows users to track project reiaifxi activities, 
generate required rqxxts, and otherwise efflciently manage the daily operations of their offices. The presentation will 
focus on the use of such programs to manage educational projects and to make project decisions based on 
accumulated data as well as the process of designing, developing, and implementing such implications. 

Frances Hensley 

The University of Georgia 

Dq)artment of Instructional Technology 

607 Aderhold Hall 

Atheas, GA 30602 
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MANAGING CONFLICT IN CBT PROJECTS 



James R. Hutton, JJi. Hutton & Associates 

As a manager, do you realize you could be spending up to 20% of your time managing conflict? Conflict occurs at 
all levels and between all participants in a CBT project This session will enable you to recognize that conflicts are 
not all negative and often have positive outcomes. You will also be able to identify elements of a conflict, sources 
of conflict, and outcomes. Participants will be asked to apply a S stq) conflict resolution process during the session. 

James R. Hutton 

J.R. Hutucn & Associates 

9841 /arpoit Blvd., Suite 700 

Los Angeles CA9004S 

(213)645-5012 



CASE MATERIALS FOR MANAGEMENT TRAINING 



Frederick G. Knirk, University of Southern California; and DeLayne Hudspeth, University of Texas at Austin 

Case study methods are excellent for many levels of computer-based training, ranging from targeted management 
issues, to high level problem-solving. This session will focus on: 1) the types of training objectives that can be 
taught by case studies; and 2) how to develop and use case materuds for those objectives. 

Frederick C. Knirk 

University of Southern California 

948 West 37th Street 

San Pedro CA 90731 

(213)743-6263 



THE COST OF NOT TRAINING WITH COMPUTERS 



Frederick G. Knirk and Mary Pinola-Kenny, University of Southern California 

While training programs are expensive, the cost of inadequately or inappropriately trained employees can cost 
MORE; die cost of NOT training can be much more than the costs of providing training. One of the best sales tools 
we have for selling CAI is to demonsttatc to our clients or employers that training is relatively inexpensive and cost- 
effective. In this session, we will demonstrate how you can use this procedure to sell your computer-based training 
programs. 

Frederick G. Knirk 

University of Southern California 

948 West 37th Street 

San Pedro CA 90731 

(213)743-6263 
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Henry //. McCaslin, Jr, and Patricia Af . Board, FBf Academy 

Computer Based Training professionals have long advocated a team ^proach to CBT develc^mcnt. Although teams 
vary in composition firom organization to organization, the commonality among them remains • ''one group working 
toward a common goal.** Building a team is not an eas> feat Team members are selected on professional experience 
and their ability to woric in a team environment. Yet is appears that one team member, the Subject Matter Expert 
(SME), is very often selected solely on the merits of **knowing a lot about the subject** 

Many masiagers unintentionally select SMEs ill-suited for the CBT development team. This often occurs because 
•here is a lack of understanding, both by managers and the SMEs themselves, on exactly what the SME's 
responsibilities arc. This presentation will focus on the role of the SME in the CBT development process. It will 
examine questions such as, does the potential SME possess personal and professional attitudes that could be 
detriment^d to the successful completion of the project? Is the SME fearful of computers? What are the benefits to 
being an SME? 

The presenters have worked on numerous projects with a wide variety of SMEs. Some were very fearful of 
computers. Some did not want the computer to **teach their class.** And others, just did not woric well in a team 
enviionment As a result of these experiences, the presenters have been able to identify characteristics and traits that 
can enhance the SME*s role in CBT development Management's awareness of these traits and characteristics can 
greatly influence the success or failure of CBT develcyment and implementation. 

FBI Academy 

Computer Basrd Training Center 
Quantico, Virginia 22135 
(703)64(Ki402 



EVALUATING TRAINING AND DEVELOPMENT PERSONNEL: 
METHODS, LEVELS. AND PiTFALLS 



William H. Wheeler, /r. The University of Georgia 

Often non-traditional in terms of process, power relationships, and outputs, the modem training and development 
function represents a real challenge to management when the productivity and contribution of personnel must be 
evaluated. An stafl^ member may be woricing on several projects, and have varying levels of responsibility which 
change on an irregular basis. Responsibility for completion of critical elements of a project often shifts to external 
organizational components. Evaluating productivity for a p^iicular person on a particular project can be difficult 
Factors such as creativity are hard to pin down. 

To manage the training and insuuctional development function effectively, the training manager must have accurate 
information. This paper will present a multiple-methodology process for c.aluating training and development 
personnel, suggest and explain appropriate levels of personnel evaluation, and point out a number of pitfalls. 

Managers often dismiss the idea of multiple methods for evaluating personnel, arguing that the process is too time- 
consuming. This paper will argue that the lack of a system that provides evaluative tracking of staff activity and 
continual formative evaluation feedback is one of the most important sources of managerial productivity loss. 
The annual evaluative hour supplemented by wecrJy yelling about unpleasant surprises may be the least efficient 
process of all. 

William H. Wheeler. Jr. 

Department of Instructional Technology 

607 Adertiold Hall 

The University of Georgia 

Athens. GA 30602 

(4C4) 542-3810 
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DEVELOPMENT OF A COMPUTER-BASED 
COLLEGE PITCH MATCHING TEST 



loanna E. Etmektsoglou, University of Illinois-Urbana'Champcdgn 

A Compuier-Based College Pilch Matching Test (CCPMT) was devel(q)ed in 1989 and included single pitches, 
melodic intervals, motives and short phrases derived from well known compositions of different styles. The test was 
administered individually to college freshman students in music education at the University of Illinois at the 
beginning of the 1990 academic year. 

The purpose of the test was to evaluate the pitch matching ability and tonal memory of the suidents. The 30-item 
test was pilot tested off-line and on-line with high school and college students before being administered to the test 
subjects. The testing completion time was 2q)proximately 12 minutes per subject A minimum of fifty subjects were 
selected for the research project In addition to the performance test, students completed a questionnaire which was 
intended to collect information on their previous musical experiences in both formal and informal educational 
settings. Test equipment included a DELL 286 computer, a Casio MT-240 MIDI Keyboard, and an AFI Pitch Board. 
The microphone input was converted to digital MIDI information fw student assessment Data were analyzed for 
accuracy with 1 cent discrimination. Sampling rate of the AFI interface was 1500 sample$/second. Students received 
evaluation feedback on accuracy of pitch performance. 

loanna E. Etmektsoglou 
University of Illinois 
URH 1066 Sherman 
909 South Fifth Street 
UrbanalL 61820 
(217)333-2250 



THE USE OF INTERACTIVE VIDEODISC IN 
ACQUIRING MUSIC TEACHING COMPETENCIES 



Mary A. Leglar and Robert W. Placek, The University of Georgia 

Most states in the U.S. now require candidates for teacher certification to pass formal competency-based tests on 
planning and presenting instruction. This has reinforced the profession's awareness of the need for field-based 
experiences in the training of teachers. However, many college music education departments find it difficult to 
schedule and supervise extensive pre-servicc teaching. 

To supplement field-based teaching experiences, a complete observation prOejam was developed using a series of 
interactive videodiscs of music teachers in actual classroom settings. Taped segments were carefully chosen to cover 
each classroom competency tested by a representative state certification instrument. The taped lessons were not 
rehearsed, and do not represent "perfect" lessons. 

Each segment was analyzed by a groiq) of experts to identify teacher behaviors which demonstrated, or in some cases 
failed to demonstrate, the required competencies. The analyses of the video segments were validated by stale 
certification officials. A computer program was developed to control the video playback equipment and guide 
individual studenis through a detailed observation and analysis of the taped lessons. Results of a pilot study suggest 
that interactive video observations improve students* ability to analyze teacher behavior, and provide an efficient 
means of supplementing pre-service field experiences. 

Mary A. Leglar 
The University of Georgia 
School of Music 
Athens, GA 30602 
(404) 542-2763 
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MUSICAT: AN ADAPTIVE TESTING PROGRAM 
TO ASSESS MUSICAL ABILITY 



Walter P. Vispoel, Don Cqffman, and Dawn Scriven, The University of Iowa 

One of ihc most significant recent advances in measurement technology is the development of computerized adaptive 
tests. Research assessing the feasibility of adaptive tests indicates that adaptive testing procedures require an average 
of 50% fewer items to yield reliabilities and validities comparable or superior to those of conventional tests (McBride 
& Martin, 1983; Moreno. Wetzel. McBride. & Weiss. 1984; Urry. 1977; Weiss. 1982). These promising results 
have led to the develqmient of the first q)erational adaptive tests which presently include adaptive versions of a) 
standardized aptitude tests, such as the Armed Services Vocational Aptitude Test Battery (Allfred & Green. 1984; 
Sympson. Weiss, & Rec. 1982) and the Differential Aptitude Tests (McBride. 1986). b) college placement tests 
(Stocking. 1987). and c) achievement tests fw elementary and high school students devel(^)ed by the Portland Public 
Schools, the Montgomery Cbunty (MD) Public Schools, and the WICAT Systems Corpwaiion (See Rudner. 1989). 
These tests are similar in that they all use visually-administered items. 

In this presentation, the features of an computerized adaptive testing program designed to assess musical ability will 
be demonstrated This program is the first operational adaptive test cif musical listening skills to be implemented on 
a microcomputer. It also is the first computerized adaptive testing program that measures a cognitive skill using 
aural rather than visual stimuli. 
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USING CD-ROM AND CBT TO DEVELOP LISTENING SKILLS 



Ernest A^oodruff and Phillip Heeler, Northwest Missouri State University 

Perceptual listening is of prime importance to the musical experience; therefore, an important goal of music 
literature and appreciation classes is to develop the listening skills of the students. The presenters have used 
principles of Keller's Personalized System of Instruction (PSI) to encourage the mastery of aund (listening) skills in 
a college music appreciation course. Implementation of PSI was accomplished by specifying aural objectives and 
then testing the mastery of these objectives by using an interactive videodisc workstation. Due to widely varying 
backgrounds and abilities, some students easily passed the aural tests at a prescribed criterion level while others were 
not able to pass aural tests after many attempts. 

The presenters are currently engaged in a research project which explores the use of an expert system in conjunction 
with the audio function of CD-ROM to help students master aural objectives. This session will describe how CD- 
ROM technology can be used to present tutorials which track learner performance and provide appropriate 
remediation. Details of the results of a pilot project will be shared with the audience. 

Ernest Woodniff 

Northwest Missouri State University 
Musk: Department 
Maryville MO 64468 
(816)662-1317 
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INTEGRATING PC-PILOT WITH EXPERT SYSTEMS: 
GRAPHICS, EDITING, AND CONSULTATION 



Ping-Cheng Chao, University of Georgia 



Most expert system development tools on the nuirket today have some sort of c^bility for disi^aying graphics, but 
tliose features provided by the tools are not sophisticated enough to increase user friendliness or offer a delivered 
system that has a polished irofessional lock, A Media Selection Expert System will be developed by using PC- 
PILOT programs to display gr^hics and demonstration programs. 



In CAI involving electrocardiogram (ECG) inierpreiation, the diq)lay of ECGs is often limited to static tracings. 
Displaying dynamic ECG tracings can significantly increase the realism and usefulness of certain types of programs, 
e.g., simulations of cardiac emergencies. 

The presentation outlines a method for simulating dya nic ECG tracings on the computer screen. The method uses 
EGA gr2^)hics, the EG APIQ utility from the PC/PILOT Advanced Feature Litaary, and a RAM disk (virtual disk) set 
up in the system configuration file. Rapid access to a large number of ECGs is possible, and each tracing can be 
displayed with continuous motion for as long as necessary. 

Participants will be shown how to set up a RAM disk and the method will be explained with reference to a sample 
ECG display subroutine, copies of which will be distributed to the audience. The presentation will include guidelines 
for keq)ing the speed of the ECG display constant with various CPUs. 



Ping-Cheng Chao 

University of Georgia 

102 College Station Rd. F-210 

Area II, University Village 

Athens G A 30605 

(404)542-4025 



SIMULATING DYNAMIC ECG TRACINGS WITH PC/PILOT 



John A. Stewart, HealthWare Corporation 



John A. Stewart 
HealthWare Corporation 
1757 Sturgus Avenue, S. #3 
SeaiUeWA 98144 
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CREATING A STRUCTURED PILOT CAi LESSON 



Marianne E. Yoder, Northern Arizona University 

For the beginning programmer, creating code in a logical, easy-lo-debug manner can be overwhelming. Although 
PILOT is a relatively straight forward authoring language, the manual which comes with PC PILOT presumes that 
the PILOT author already understands the logic of programming. PILOT is forgiving, so that loosely written 
programs will run properly. However, such programs may prove difficult to modify when further testing of the CAI 
program demonstrates the need for modification of the instructional presentation. This presentation will demonstrate 
several strategies the beginner programmer can use to produce structured PILOT code. The i«)gramming techniques 
demonstrated will help organize the CAI lesson into manageable sections, decrease the amount of time necessary 
writing code, improve the readability of the code when future revisions are necessary, and make debugging easier. 
Code from a sample ptogmn demonstrating a menu, random selection of questions, ond pop-up dialogue boxes will 
be available. 

Marianne E. Yoder 

1496 W. University Heights Drive North 
Flagstaff, AZ 86001 
Home (602) 779-5535 
Work (602) 526-2671 
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LESSONS LEARNED IN APPLYING NOVANET TO ALTERNATIVE EDUCATION 
(BASIC SKILLS DEFICIENT AND DROPOUT STUDENTS) 



A. Sidney Alpert, University Communications » Inc. 

Various schools and programs in and around the country have now had four years of experience in using NovaNET to 
solve the ever increasing high school dropout and basic skill deficiency problems (alternative education). This 
presentation describes the various problems encountered, the existing NovaNET based solutions, the historical 
results, and suggestions for improving NovaNET implementation in these areas. 

This presentation will include the following lopics; a) Sites and Programs; b) Facts and Figures; c) ImpleL\entation 
Structures; and d) LesscHis Learned 

A. Sidney Alpert 
University Conununications 
1465 Post Road Street 
WestportCt 06880 
(203)255-6044 



INNOVATIONS IN TRANS-TUTOR 



Richard W. Blomme 

New ideas associated with the development of the experimental Trans-TUTOR system will be discussed. Focus will 
be on the underlying structure of the language and changes which make it simpler and more flexible than the 
TUTOR language from which it evdved. Issues associated with transportability of material will also be discussed. 

Richard W. Blomme 
lllE.Hcaley,Apt 304 
Champaign IL 61820 
(217)333-7523 
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HOW TO DELIVER INDEPENDENT STUDY 
(CORRESPONDENCE) COURSES VIA NOVANET 



Bruce A. Cameron 



As alternative education looks for solutions to graduating dropout and remedial high school students, an open entry, 
open exit, self paced method of getting student through course requirements is needed. Because traditional 
correspondence courses have problems motivating students and providing interactive instructor contact, many 
students do not succeed. This presentation will demonstrate the feasibility of successfully delivering interactive 
independent study courses to students via NovaNET. 

Topic covered in this presentation will be: 

Prerequisites to Developing an Interactive Independent Study Course, including instructors, learning 
objectives, scope and sequence, textbooks, learning activities, £^)propriate NovaNfET lessons, assessment 
and competency testing questions, and grading; 

• Determining How to Implement Learning Activities, both on line and off line, and the strengths and 
weaknesses of the computer in accomplishing specific learning activities; 

• C-router (An Open Architecture Student Management System), including syllabus entry, assessment 
sequencing, and student data management; 

Student Teacher interaction, including student/teacher electronic mail, live conversations, and monitor mode 
(As a "networked system", courses can be delivered over long distances and under abnormal circumstances 
such as for homebound students. Location of students and teachers is immaterial), and interaction with other 
students in the course; 

Creating New on Line Materials Specific to the Course, such as Testing 123 (on line tests), CFP drills (the 
distinction between rote and conceptual learning), and developing your own aitorial lessons. 



TTie purpose of the NovaNET Showcase is to provide an opportunity for persons using NovaNET or PLATO to 
demonstrate courseware, ctirricula, software, use of peripherals, programming techniques, etc. Each presentation will 
take place at a terminal and will be 10 to IS minutes long, with time for questions. Presentations will be concurrent, 
and will be repeated. 



Bruce A. Cameron 

3895 N. Business Center Drive 

Suite 101 

Tucson AZ 85705 

(602) 888-3076 



NOVANET SHOWCASE 



Celia Kraan, University of Illinois 



Celia Kraau, 
University of Illinois 
CERL-252 ERL 
UrbanalL 61801 
(217) 244-4296 
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DESIGNING THE USER INTERFACE AS GENRE: 
RESTRICTING CONTEXT FOR MEANING AND BEHAVIOR 



Terence C, Ahem and Jamie Myers, The Pennsylvania State University 

The ci^acity of short-tcnn human memory requires that large amounts of information be linked by indexing terms or 
experiences that become implicit refexnts. These implicit referents allow humans, when confronted by novel 
situations, to model the new uncertainty by creating analogies or metaphors from previous experiences. Developers 
of courseware, therefore, must be aware that any instructional context such as CAI will evoke implicit referents in 
the learner. These referents provide the learner a model of the appropriate suucture of lesson content and interactive 
behavior. 

Terence C. Ahem 

The Pennsylvania State University 

201 Chambers Building 

State College PA 16802 

(814)865-0624 



USING CAI TO INCREASE FLEXIBILITY IN PROBLEM*SOLVING 



Terence C. Ahern and Fernando Senior, The Pennsylvania State University 

Research has found that a learner's ability to solve problems can be constrained by functional fixedness, habit, or 
even their world view. This study assessed the effectiveness of an alternative type of courseware designed to increase 
cognitive flexibility in problem-solving. It was hypothesized that if learners had opportunities to solve reading 
problems from a variety of theoretical perspectives, their flexibility in problem identification and solution would be 
enhanced. This software also incorporated an alternative model of assessment by providing learners with open-ended 
problems and sample expert solutions in the teaching of word identification skills. 

Terence C. Ahem 

The Pennsylvars. State University 

201 Chambers Building 

State CoUege PA 16802 

(814)865-0624 



SCREEN DESIGN: LOCATION OF INFORMATION 
AND ITS EFFECTS ON LEARNING 



Macarena Aspillaga, Westlake School: and Zhongmin Li, Utah State University 

This study examined the effects of screen location of information displayed in CAI lessons on the retention of 
information. Sixty undergraduates were randomly assigned to one of three text display treatment conditions: 1) text 
display relevant to graphical information; 2) text display at the upper middle section of the screen; and 3) random text 
display. Results indicated that displaying text at a consistent location or relevant to graphical information facilitated 
learning. 

Macarena Aspillaga 
Westlake School 
700 North Faring Road 
Los Angeles CA 90077 
(818)440-1537 
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PARAPHRASING IN CAI 



Catherine Augustine and Kyle L. Peck, Pennsylvania State University 

Traditional CAI generally involves a learner responding to objective-based questions requiring multiple choice or 
short answer responses. However, paraphrasing strategies are emerging as a potentially important tool for CAI 
designers. Periu^s, as Jonassen (1988) proposes, paraphrasing forces the learner to process information in a way that 
requires more "generative processing" of »nformaiion, leading to increased retention. 

This study investigated paraphrasing in CAI and its impact on retention, comparing paraphrasing to "short-answer" 
and control conditions. Sixty-five university student volunteers were divided into three groups. Thirty comparable 
instructional objectives were randomly assigned to three sets. A balanced design (illustrated below) caused one-third 
of the students to paraphrase objective-related content for each set of items, to respond to short-answer questions for 
another set, and to perform no additional processing on a third set Therefore, each of the thirty objectives was 
attempted by all 65 students (one-third under paraphrasing support, one-third under short-answer questions, and one- 
third with no additional processing) and all 65 students participated in each condition. 



Question Set A 
Question Set B 
(Question Set C 



Group 1 
Paraphrase 
Short Answer 
Control 



Group 2 
Short Answer 
Control 
Paraphrase 



Results support the use of paraphrasing in CAI. 



Group 3 
Control 
Paraphrase 
Short Answer 



Catherine Augustine 

The Pennsylvania State University 

503 West Drive 

Boalsburg PA 16827 

(814)863-1691 



A STUDY OF THE EFFECTS OF ORGANIZATIONAL CUES, 
GLOBAL ITEM PLACEMENT, AND ADMINISTRATION MEDIUM 
ON ITEM RESPONSE PATTERNS OF 
A COURSE EVALUATION INSTRUMENT 



Mary J. Boyce, Texas A&M University 

In this study the effects of organizational cues, global item placement, and administration medium on student 
responses to questions was investigated The questions were pan of a course evaluation instnimenL It was found that 
students do answer questions differently when completing the evaluation on a computer, and when provided 
organizational cues. 



Mary J. Boyce 
Texas A&M University 
Educational Technology Program 
College Station TX 77843 
(409) 845-7185 
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A MODEL FOR COMPUTER-MEDIATED COOPERATIVE LEARNING: RESEARCH 

FINDINGS AND ISSUES 



David W, Dalton, Kent State University 

This presencadoa will summarize researdi findings related to computer-based cooperative learning and propose an 
agenda for future researcit The fdk)winggeoeial areas 1) The deGnitiofiofcooperative learning; 2) 

The ratkmale for implementing cooperative programs; 3) Research finding; 4) A taxonomy for classifying current 
and future research projects; and S) A review of methodologies and dq)endent variables /or research in computer- 
roediaied cooperative learning. 
i 

David W. Dalton 
Kent State University 
40S White HaU 
Kent OH 44242 
(216)672-2294 



THE RELATIONSHIP AMONG LEARNING STYLES, ATTITUDES, AND STUDENT 
PERFORMANCE IN A COMPUTER APPLICATIONS COURSE 



Gayle V, Davidson and Wilhelmina C. Savenye, The University cf Texas at Austin 

While researchers have studied the effects of students' learning styles and attitudes toward computers, to date there has 
been little research on the relationship between these variables. This presentation will describe the rationale, 
methodology* and results of a study ^ch extends learning style research by investigating the relationship among 
learning styles, attitudes, and performance in a computer applications course for prcservice teachers. 

Gayle V. Davidson 

The University of Texas at Austin 

Dqnrtment of Curriculum and Instruction 

EDB406 

Austin TX 78712-1294 
(512)471-5211 
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THE RELATIONSHIP BETWEEN TASK ANALYSIS 
AND KNOWLEDGE ACQUISITION 



Charles R. Dills and Alexander Romiszowski, Syracuse University 



Abstract 



Tbe fields of knowledge acquisition and task analysis have not been well-integrated, and not much cross-fertilization 
has so far occurred The authors explore the possibilities for such integration, pointing out how each field can benefit 
from the other, and i(tentifying kinds of research which would benefit both areas. The alternative meanings of 
"knowledge acquisition** as used in the two fields are clarified and contrasted. 



The authors argue that knowledge acquisition, a branch of computer science, and task analysis, a branch of 
instructional development, have much in common. After presenting some preliminary arguments to this effect, the 
authors list ways in which task analysis could benefit from an adoption of current knowledge acquisition methods 
and ways in wMch knowledge acquisition could benefit from a greater knowledge of task analysis and the adoption of 
many of its methods. 

The possibility of other benefits fiom cross-fertilization between instructional development and computer science is 
discussed Chief among these is the development of new ways to structure knowledge. Research on intelligent tutors 
has shown ways in which knowledge can be structured so as to represent optimal learning patterns for interactive 
instruction. 

There are also some dangers in the unsystematic fusion of instructional development and computer science concepts 
and methodologies. The authors have made a distinction between laiowledge acquisition" in the task analysis sense 
and *laK)wledge acquisition** as learning that points out how loose usage of the terminology often leads to confusion. 
As an example of such practical confusion, the recently-coined concept of "situational cognition" is analyzed. 
Implications for research and development in both ihe fields in question are discussed. 



Summary 



Charles R. Dills 
Syracuse University 
330 Huntington Hall 
College of Education - IDD&E 
Syracuse NY 13244-2340 
(315)443-3703 
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LEARNING FROM HYPERMEDIA: 18 THIS AN INFORMATtONi INSTRUCTIONAL, OR 

LEARNING MEDIUM? 



Barbara LGrabawdi and Ruth V. Curtis, Syracuse University 

Hypert^ was defined by Nelson (1960) as a 
docomeats thai guide leaders fiom (me to aiKMh^^ 

"lof^cal extenskm oi hypertexu encompassing grq>Iiics, animation* video* soond, and so on** (Osgood, 1987). 
Because (tf a tremendous increase in infonnation t^ 

created, developing hypermedia databases has become a very attractive means for presenting diis information. 
However, herein lies a serious issue. What is being passed off as instructional may, in tact, be simiriy information. 

IttfonnatioD Ifiks been described as a process, a flow c{ messages (MIson, 1988) with many functions, one of 
which is to instruct (MacMuUin and Taylor, 1984). Instmction, on the other hand, may be defined as a way of 
organizing and sequencing information for die leanm^ which may include any one or all (tf a number of essential 
elements such as presentation of mformation and provision of examples, practice and feedback ^igeluth, 1983). 
Leaminf results Crom active cognitive imcessing of infimnation. However, infcmnation is not always 
instructional, nor does viewing information always result in learning. Rnally, instruction does not always cause 
learning nor does leamii.g always require instruction. Fleming and Levie (1978) classify the difference httwren 
instruction and learning according to the locus of die activity; dial is, instruction occurs outside die learner vMc 
learning is the result of internal processes widun the learner. The relationship among diese phenomenon may be 
represented 1^ the diagram betow. Tne shaded area rqpresents die convergence of die A 




From this conceptual mode, we will discuss factors to consider in the design of a hypermedia 

stack for information, instructional or learning purposes. These bcto^ can also be used for evaluating existing 

hypermedia **lessons" to determine if tfiey are, indeed, informational, imtructional or learning tools. 

Referencas 

Heming, M. and Levie. W.H. (1978). Instructional Message Design, Englewood Cliffs, NJ: Educational 
Technology Publications. 

MacMuIlin, S.E. and Taylor, R.S. (1984). Problem Dimensions and Information Traits. The Information Society 
5(1), 91-111. ^' 

Osgood, D. (1987). The Difference in Higher Education: Five Colleges Meeting the Challenge! Byte, 12(2\ 165-68, 
170. 172-178. 

ReigeluUi, C.R. (1983). Instructional Design Theories and Models: An Overview cf Their Current Status, Hillsdale, 

NJ: Lawrence Eribaum Associates. 
Wilson, P. (1988). Mission and Information: What Business are We In? Journal of Academic Librarianship, 14(2), 

82*86. 
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Syracuse University 
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CBi TECHNIQUES FOR ENCOURAGING ACTIVE MENTAL PROCESSING 



Barbara L. Grabowski and Eileen Schroeder, Syracuse University 

An important current focus of research is examining instructional and learning strategics from two perspectives: one 
which affects the organization of information to be learned, and the second which affects who does the organizing-- 
the instruction itself, or the learner. From the first perspective, research has been conducted mainly in two broad 
areas: one which relates to-be-leamed information to existing cognitive structure and the second which affects uic 
structure and organization of the information. The theoretical basis for this research is based on cognitive theory, 
information processing, and schema theory (Mayer. 1979. Rumclhart. 1980). Examples of instructional/learning 
strategics which are influenced by this research include the use of analogies, concept maps, pattern notes, use of 
examples or main ideas, to name only a few. The second perspective involves the question of who does the 
connecting or organizing of information-the insUruction or the learner. Researchers are investigating the effect of 
providing the link or organization of the information for the learner in a supplanted, instructional way. vs. the effect 
of having the learner generate those links for him/herself. In 1966. Rothkopf began advocating the use of 
mathemagenic (those that give birth to learning) strategies that evoke active cognitive processing (see also 
Rothkopf. 1970). Following this reasoning. Wittrock began writing about generative learning (Wittrock. 1974a, 
1974b). In a generative mode, learners accept responsibility for examining the underlying structure of the 
information to be learned and generate their own relationships among concepts and to their own knowledge 
(Wittrock. 1974a. 1974b. 1985). Much research exists to support the positive effects of both supplanted and 
generative strategies, but when compared one to the other, while research res jlts arc mixed, often generative 
strategies show more learning gain. 

In an attempt to understand and explain the mixed results, we hypothesized that the selection of a specific generative 
or supplanted strategy which links to-be-learned information lO existing cognitive structure should be determined by 
the cognitive processing abilities of the learners. Learrjmg strategies were examined for cognitive processing 
requirements and then matched with cognitive strengths of the learner based on various cognitive styles. 

This research should influence the development of new models for computer based instruction from a number of 
perspectives. First of all. it frees us from a solely tutorial based model of CBI and provides much support for newly 
developed computerized "learning tools" such as Kozma (1989) has developed. These "learning tools" facilitate the 
manipulation of information by removing the hassles of manual erasing/redrawing. The second is from the 
perspective that the computer can be programmed to diagnose styles and automatically branch to a learning or 
instnictional approach that matches the cognitive strengths of the learner. The question we attempt to answer in this 
presentation is whether these learning tools which provide opportunities for facilitated generative learning are 
appropriate for all learners, or whether there are strategies that *vould be best presented in a supplanted, instructional 
manner for specific individuals. 

A decision matrix for selecting supplanted vs generative strategies will be presented and elaborated upon using 
analytic and global, and serialistic and holistic cognitive styles matched with two learning strategies of main Ideas 
and concept maps. Results of a pilot study which used a computerized tool examining these hypothesized 
relationships will be presented 

References 

Kozma. R.B. (1989). Principles Underlying the Learning Tool. Paper presented at the annual i«eeting of the 

American Educational Research Association . San Francisco. CA. March 27-31. 
Mayer, R.E. (1979). Twenty Years of Research on Advance Organizers: Assimilation Theory is Still the Best 

Predictor of Effects. Instructional Science. 8. 133-167. 
Rothkopf, E.Z. (1966). Learning from Written Instructive Material: An Exploration of the Control of Inspection 

Behavior by test-like events. American Educational Research Journal. 5(4). 241-249. 
Rothkopf. EZ. (1970). The Concq)t of Mathemagenic Activities. Review of Educational Research. 40(3), 325-336. 
Rumelhart. D. E. (1980). Schemata: The Building Blocks of Cognition. In RJ. Spiro. B.C. 
Bruce, & W. F. Brewer (eds.). Theoretical Issues in Reading Comprehension: Perspectives from Cognitive 

Psychology, Linguistics, Artificial Intelligence and Education. Hillsdale. N J.: Lawrence Erlbaum Associates. 
Wittrock, M.C. (1974a). Learning as a Generative Process. Educational Psychologist. 77(2). 87-95. 
Wittrock. M.C. (1974b). A Generative Model of Mathematics Education. Journal for Research in Mathematics 

Education. 5(4), 181-196. 

Wittrock, M.C. (1985). Teaching Learners Generative Strategies for Enhancing Reading Comprehension. Theory 
into Practice. 24(2), 123-126. 



Selected Absnacts 



BaibnL.Grabowsid 
Syiiaise University 
330 Huntington Hall 



Cdlege of Education • IDD&E 
Syracuse NY 13244-2340 
(315)443.3704 



THE MODEL OF A PEDAGOGUE IN A COMPUTER-AIDED ELEMENTARY SCHOOL 

SCIENCE LESSON 



The need for courseware authors to attend to the issues of pedagogy has long been debated (Hannafin & Mitzel, 1990; 
Merrill, 1985; Zajonc» 1984; Roblyer, 1981). While pedagogy can play a critical role in learning new information, it 
is a costly lesson requirement and is dius often neglected in the courseware development process. The 
microcomputer-based pedagogue presented here is a demonstradon of a cost-effecdve and pedagogically efOcient 
model. 

This computer-based pedagogue is an embedded lesson feature designed to acdvely engage die student in lesson 
content by employing an inquiry style of teaching. This model uses cognitive instructional techniques to guide 
student interest via a series of activities ranging from information gadiering through hypodiesis testing. The lesson 
content focuses on elementary school science for grades 3 through 5, presented in a student-centered exploratory 
science lesson. 

Reference!! 

Stevens, A. L. & Collins A. The structure of a Socratic tutor. Proceeding of die Association for Computing 

Machinery National Conference, Seatde, Washington, 1977. 
Hannafin, K. M. and Mitzel, H.E(in press). CBI Audioring Tools in Postsecondary Institutions: A Review and 

Critical Examination. Computers and Education: An International Journal. 
Merrill, M. D. (1985). Where is die audioring in audioring systems? Journal of Computer-based Instruction^ 12(4), 

90-96. 

Roblyer, M.D. (1981). Instructional design versus audioring of courseware: Some crucial differences. AEDS Journal, 
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Zajonc, A. R(1984). Computer-pedagogy? Questions concerning die new educational technology. Teachers College 
Record, 85(4) 569-511. 



Kathleen McDermott Hanncfin and James £. Gall, Florida State University 
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THE DESIGN AND DEVELOPMENT OF A COMPUTER-BASED 
LEARNING SKILLS PROGRAM: AN EXAMPLE OF INSTRUCTIONAL AND 
MOTIVATIONAL DESIGN MODELS IN ACTION 



Atsusi Hirumi, Florida State University 

The purpose of this presentation is to describe the application of both instructional and motivational design models 
in the development of a computer-based learning skills program. The objectives are to provide instructional designers 
and courseware develq)er$ with: 1) a systematic method of applying instructional and motivational theory during the 
design and development of computer-based instruction; and 2) examples of how instructional and motivational 
strategies may be applied in the development of learning skills courseware. A theoretical foundation for integrating 
instructional and motivational strategies during the prq)aration of computer-based instruction is also discussed. 

Atsusi Hirumi 

Center for Educational Technology 
Florida State University 
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TOWARDS A MULTIFACETED THEORETICAL FRAMEWORK FOR COMPUTER- 
ENHANCED LEARNING ENVIRONMENTS 



\v^ile there has been a dynamic growth in the utilization of instructional computing technologies over the past 
quarter of a century, the articulation of the theoretical basis of these technologies in the fields of education and 
training has been more irotlematic. This presentation suggests that-in the near future at least-no single theoretical 
paradigm will be capable of guiding the development of instructional computing systems for the wide range of 
learning situations that exist. Instead, a multifaceted theoretical framework is being proposed to deal with a 
pluralistic theoretical universe. 

This frameworic allow the analysis of selected cognitive theories of learning which have particular relevance to the 
development of different types of computer-enhanced learning environments. In particular, the fran^work focuses on 
two critical elements associated with the overau learning context: (a) conceptual characteristics of the domain being 
learned (i.e., the complexity of the concq)ts and tasks and the degree of hierarchical structure within the domain), and 
(b) the learner's stage of learning (i.e., novice or advanced). The multifaceted theoretk:al framework is intended to 
offer a meta-level perspective for analyzing the complex interaction of cognitive theoretical constructs, contextual 
elements associated with learning, and characteristics of different types of computer-enhanced learning environments. 
It is hoped that such a framework could provide both analytical and practical guidance for applying the most cunent 
cognitive instructional theories and research to the increasingly sophisUcated arsenal of emerging technology 
systems. 



The Educational Resources Information Center provides access to a wealth of information in the field of education. 
This presentation will cover what is available and an overview of the different access points to the infomiation. 



Michael /. Jacobson and Jihn-Chang Jehng, University of Illinois at Urbana-Champaign 
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ERIC: WHAT IS IT AND HOW TO USE IT 



Jane Klausmeir Janis. ERIC Clearinghouse on Information Resources 
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THE EFFECTS OF AUDIO IN OBI ON READING 



William E, Johnson and Gregory C Sales, University of Minnesota 

Audio, partkulaiiy speech, is becoming a common .eature of computer-based instruction. Yet, little research on the 
effectiveness of this feature has been conducted. This presentation examines results of a research project which 
investigated the effects of audio and speech features in a popular commercial phonics and vocabulary product 

William H. Johnson 

University of Minnesota 

235 Rut HaU, Pillsbuiy Drive SE 

Minneapolis MN 55455 

(612)626-7498 



CONVEYING STRUCTURAL KNOWLEDGE 
IN HYPERTEXT KNOWLEDGE BASES 



DandH. Jonassen and Sherwood Wang, University of Colorado 

One of most often cited putative advantages of hypertext is that it can mimic the associative properties of human 
memc^. That is, the nodes and links in a hypertext knowledge base are able to map the active structural networks 
(Quillian, 1968) ox semantic networks that deiscribe die constructions of human memory. The implicit hypothesis of 
this advantage is that knowledge structured hypertext will map more readily onto users* knowledge structures. 

Another way of conceptualizing these semantic properties of memory and hypertext knowledge bases is structural 
knowledge. Structural knowledge is the understanding of the interrelationships between ideas in memory. It is a form 
of concqxual knowledge that mediates the gap between declarative and procedural knowledge. Declarative knowledge 
represents awareness of ideas, while procedural knowledge represents application of that knowledge. Procedural 
knowledge, however, is predicated upon structural kncwledge in order to fulfill the functional relationships between 
ideas. 

If hypertext is better able to represent structural knowledge in its knowledge base, the issue is how best to convey 
that structme the user interface. Jonassen (1990) described a vaiiety of tools for eliciting structural knowledge 
from experts in order to map onto hypertext structures. These techniques explicitly map ideas and their structural 
relationships. These techniques were used in this study to map the structure of knowledge onto a hypertext 
knowledge base about hypertext 

Methods 

Instromeiits* In order to structural knowledge acquisition, learners completed a word association task prior 
to treatment consisting of free word associations to key concepts related to hypertext Free word association is the 
benchmaric method for assessing cognitive structure (Preece, 1976). Following treatment, learners once again 
completed the same task. oUowing the free word association, learners were administered a multiple choice test which 
pan^phrased the nature of the relationships between sucteen key terms related to hypertext These are accepted tests of 
structural knowledge. Additionally, learners completed recall tests of information contained in the knowledge base. 

Treatments. Three treatments were developed for this study from the HyperCard version (Jonassen, Roebuck & 
Wang, 1990) of the book Hypertext/ Hypermedia (Jonassen, 1989). The hypertext was a browsing system consisting 
of 240 card m three stacks with bookmaricing and limited annotation capabilities. Treatment 1 consisted of embedded 
referential links between cards. Terms in the text were highlighted, enabling learners to traverse the links. Treatment 
2 consisted of the same capabilities with the added feature of a pop-up window mediating each link. Text in the 
window stated the explicit relationship between the node the learner was leaving and the node the learner was going 
to. Treatment 3 consisted of the same card with no embedded links. Instead, in order to navigate, learners had to 
access a graphical browser, which visually mapped the node and all of the possible links to it Overview maps 
showed all of the nodes in a section of the hypertext. 
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Frocedure* Prescrvice teachers in a teacher education program were given the task of learning about an important 
new instructional technology, hypertext, as an assignment in a pre-scrvice instructional technology course by using 
a hypertext. This was a required assignment in their course. In a Macintosh laboratory, students individually 
interacted with the hypertext in order to acquire as much information about this new technology as possible. A 
monitoring program was added to the stacks to audit the action and the time allotted to each learner action. 
Following the interaction, learners were asked to complete the posttests. 

Results 

Data arc being collected in iur: September of 1990. It is anticipated that learners in the graphical browser group 
will exhibit the highest level of structural knowledge. That is, they will generate the greatest number of 
associations. It is anticipated that learners in the p;.p-up link explanation version will perform better than the 
referential link version on the multiple choice test. Multivariate analysis of variance will be used to analyze 
difference between group performances. Audits of learners' navigations will be entered into transformation matrices 
which win be analyzed using cluster analyses. 
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COMPUTER-BASED INSTRUCTION IN THE DEVELOPING COUNTRIES: 
A RESEARCH-BASED MODEL FOR DEVELOPMENT 
AND IMPLEMENTATION 



Marshall G. Jones, Steven W. Harmon, Thomas C. Reeves, University of Georgia 

This study reports on the status of CBI in 10 developing countries. The findings arc based on interviews with 
national educational leaders plus interviews with US consulianls. The findings form the basis for a CBI development 
and implementation model in developing countries including: strategies for dealing with cultural sensitivity, 
technological appropriateness, and conviviality. The roles of donor organizations and international consultants arc 
also critically examined. 

Marshall G. Jones 

Depattment of Special Education 

570 Aderhold Hall 

The University of Georgia 

Athens, GA 30602 

(404) 542-4770 
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PRESCRIPTIVE STRATEGY ANALYSIS: PROGRESS REPORT OF THE SECOND 
GENERATION ID RESEARCH PROGRAM 



Mark K, Jones and M. David Merrill, Utah State University 

The overall goal of the Second Generation Instructional Design Research Program is to develop a comprehensive 
new generation of instructional design 'Jieory and methodology, and to build a set of intelligent tools that aid the 
designer in applying the theory. In the present project. Prescriptive Strategy Analysis, the researchers are developing 
the theory and methodology for automated prescription of instructional strategies, and will implement a prototype 
Strategy Analysis Component (SAC) intelligent tool to carry out this function. 
SAC performs the following functions: 

• gather generally applicable information about the audience and the resources and constraints of the setting, 
ani storing this in strategy knowledge base; 

• recommend instructional transactions to instruct content identified by the knowledge analysis; 

• direct and constrain further knowledge acquisition based on recommended transactions; 

• prioritize authoring of transactions. 

The session will overview the goals and design of the project, and discu the impact of this approach upon the 
instnictional development process. 

Mark K. Jones 

Utah State University 

Department of Instructional Technology 

Logan UT 84322-2830 

(801) 750-2630 



COMPUTER-BASED INTERACTIVE VIDEO: 
POTENTIALS AND LESSONS FROM PAST RESEARCH 



Karen Lee Jost, Syracuse University 

Past media research tried to show that the medium itself made a difference. Research in insuuctional technology 
needs to be directed, not at the medium, but at such things as instructional methods, the settings in which 
instruction takes place, the cognitive processes engaged through the use of symbol systems, the effect of how 
content is structured, and the use of learning strategies. It is also important to explain how individual differences 
affect how learners process and carry out insuructional activities and to investigate the impact that various kinds of 
interactions can have on different learners. Refmed questions focusing on specific salient attributes and qualities of a 
medium, such as the depiction of movement, also furnish us with important information which can be applied to the 
design of effective instruction. 

Computer-based interactive video is a complex combination of technologies. In order to fully understand its 
attributes, aq)abilities, and potentials we need to look at its many component parts. From an educational standpoint, 
we are not only interested in how they have been typically used, but in what they can potentially be made to do. The 
purpose of this presentation is to analyze the interactions between learner characteristics and instructional features. It 
will address the unique capabilities and potentials of CBIV. The presentation will also discuss important lessons 
from research on other media and on specific salient media attributes that we can build on in understanding how these 
features affect the learners* cognitive processes when interacting with CBIV. Included will be a discussion of the 
effects of such insuuctional features as learner control, learning sti^tegies, and motivational variables. 

# 

Karen Lee Jost 
Syracuse Univ^ity 
330 Huntington Hall 
College of Education - IDD&E 
Syracuse NY 13244-2340 
(315)443-2404 
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INTELLIGENT TOOLS FOR INSTRUCTIONAL DESIGN 



Zhongmin Li oirf Mark K. Jones, Utah State University 

The goal of the Second Generation Instructional Design Research Program is to develop a new generation of 
instructioaal design theory, and to build intelligent tools to aid designers ^)plying the theory. This roundtable will 
discuss some of the issues raised by research into developing intelligent tools for instructional design, such as their 
feasibility, the utility of techniques developed for experts systems construction, and the value of tool building as an 
adjunct to Aeoiy development 

Zhongmin Li 

Utah State University 

DqMvtment of Instructiorud Technology 

Logan UT8432^2830 

(801)750-2630 



KNOWLEDGE ACQUISITION AND ANALYSIS: PROGRESS REPORT OF 
THE SECOND GENERATION ID RESEARCH PROGRAM 



Zhongmin Li and Af . Da\id Merrill, Utah State University 

The goal of the Second Generation Instructional Design Research Program is to develop a new generation of 
instructional design theory, and to build intelligent tools to aid designers applying the theory. In tbt present project, 
the researchers develop a theory for acquiring, analyzuig, and representing subject matter knowledge for instructional 
design and delivery, and develop an intdligent tool to perform this function. This session overviews the goals and 
design of the project, and demonstrate the system. 

Zhongmin Li 

Utah State University 

Dqiartment of Instructional Technology 

Logan UT 84322-2830 
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FEEDBACK INDUCED COGNITIVE DISSONANCE 
AND ITS EFFECT ON LEARNING AND RETENTION 



Joseph A. hfarrone and Kyle L Peck, Pennsylvania State University 

Although algorithms exist to allow computers to evaluate open-ended answer^ during CAI, this evaluation is far 
from perfect However, attempts to make computers baicr evaluators may be misguided. By asking computers to 
evaluate answers we may be eUminating a powerful tool in the learning process. 

This study investigated the beneHt of asking students to validate the computer's imperfect evaluation of their 
answers, speculating that the cognitive activity involved in examining and contrasting the preferred answer and the 
one considered incorrect may enhance learning and retention. 

Following video-based instruction, one hundred and seven university students took a test consisting of 24 open-ended 
items. Half of the questions employed intentionally inferior answer analysis, increasing the probability that 
potentially correct answers would be considered incorrect After answering all 24 items, each item was reviewed with 
the student a^d students were allowed to agree or disagree with the computer's assessment and, in one treatment, to 
provide a written explanation of why their answer was correct The dissonance and subsequent processing caused 
when the computer incorrectly evaluated a correct response as incorrect led to significantly greater retention as 
measured on an identical test administered one week later. 

Joseph A. Marrone 
Pennsylvania State University 
248 Chambers Building 
Stale College PA 16802 
(814)865-2161 



ESTABLISHING AND MAINTAINING A PROFESSIONAL KNOWLEDGE BASE 
THROUGH EXTENDED COMPUTER-MANAGED PRACTICE 



Kyle L. Peck, Pennsylvania State University 

Salisbury (in Jonassen, 1990) provided recommendations for CAI drill designers based on cognitive learning theory: 
automaticity; interference; spaced practice; spaced review; and capacity of short term memory. A computer-managed 
practice system based on these principles was denK)nstnued 

Practice items move through six "stales,** from **ncw** through 'rehearsal,** **drill,** and three reviews. After successful 
completion of an item it advances to the next state, is removed from the **working pool** of items and is **date 
stamped** for retrieval during a subsequent session. The duration of delays between sessions may be set by the student 
(or researcher) as may the number of items in the working pool, whicn determines the size of the subset of items 
presented concurrently, limiting interference and accounting for limitations of short tern memory. 

This system, based the **progressive state** model proposed by Atkinson (1974), is capable of presenting activities of 
^ypc* including complex labeling activities and simulations. 

Research examining the size of the working pool reveals that although there are no significant differences in 
achievement when the working pool is increased from 7 to 14 items, students voluntarily using the lessons on their 
own time remained engaged in the activity longer when working with a 14 item working pool. 

KyleL. Peck 

Pennsylvania State University 
248 Chambers Building 
State College PA 16802 
(814)865-2161 
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INSTRUCTIONAL THEORY INTO PRACTICE: INSTRUCTIONAL MODELS 



Franklin C. Roberts, Western Washington University; Peter J. RizzCy Ptinceton Center for Education Services; 
Thomas Reeves, University of Georgia; and Lynn Misselt, Authorware, Inc, 

Designing effective instruction is a difficult, labor intensive activity. Deiq^ite elaborate training and new software 
tods, we often reinvent the wheel when we design instruction. We fail to capitalize on our experience, and we often 
ignore the commonality that exists across many training and instruction tasks. One way of overcoming this is 
through the design and use of instructional models. This notion of instructional models provides a potentially 
powerful avenue by which effective instructional materials can be developed more quickly and more easily than by 
traditional methods. 

This panel session brings together people working on four iiffcrcnt projects in instructional modeling. The first uses 
models in a graduate program in insuuctional technology. In a course on CBI, a model has been designed which 
provides a generalized approach to the design of mtorial and drill lessons. This model focuses the students' attention 
on manipulating a set of micro-strategies that reflect principles of instructional design and screen management 

The second project used insuuctional models in the development of a large curriculum project This project required 
the development of numerous CBI lessons in an academic curriculum. A small library of models were developed that 
reflected common strategies that were found across the curriculum. This models-based I9)pix>ach was used to make the 
design task easier, and to ensure that the final lessons were both consistent and pcdagogically sound. 

The third project is an effort to develop a set of models for use within an authoring system, Authorware 
Professional. This has resulted in a library of model-based productivity aids. The models arc an i^lication within 
the authoring system, so that the user can rapidly build lessons by organizing a set of pre-exisUng model 
components. The lessons can be modified to reflect different strategies or content. The facility is also available for 
users to build their own set of models. 

The final project considers models in instructional technology research. One of the perennial problems in social 
science research is the lack of rcplicability. In instructional technology research, studies are often conducted over 
extremely short periods of time, or with unvalidaied materials, or with fictitious content This project imposes the 
development of a set of standardized lessons across a range of curricula. The project would develop and validate a set 
of instructional models, and then implement them across several content areas. Researchers could then manipulate 
various components in the materials, allowing greater so-ength in their conclusions and in the gencralizability of the 
results. ' 



KNOWLEDGE ACQUISITION SYSTEMS BASED UPON STRUCTURED 
COMMUNICATIONS AND PATTERN NOTING TECHNOLOGIES 



Alexander Romiszowski and Charles /?. Dills, Syracuse University 

Structured Communication and Pattern Noting arc both educationci tools that have recently become popular. Some 
experiments exploring their development as knowledge acquisition tools, both independently and as an integrated 
system, arc described, and ways in which they can be used to acquire decp-stiucture knowledge are described. 

Alexander Romiszowski 
Syracuse University 
330 Huntington Hall 
College of Education - IDD&E 
Syracuse NY 13244-2340 
(315)443-2404 
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ENHANCING PERFORMANCE THROUGH 
HUMAN AND AUTOMATED COACHING 



Allison Rossett, San Diego State University 
What makes an effective human or automated coach^ 

the differences between off-line and on-line coaching (9)portunities mean to someone who is designing a course* 
supporting a colleague or developing computer-based training? This presentation will use research, theory, and lively 
examples to coach the audience on coaching. 

Allison Rossett 

San Di^ State University 

Dqmtroent of Educational Technology 

San Diego CA 92182-0311 

(619)594^?!§ 



COMPUTER ANXIETY: OVERVIEW 



Mioko Saito, University cf Oklahoma 

Computer-based instniction CBI has been introduced to schools to provide more effective and efficient eam:n. As of 
1990, there are over 20,000 educational software tides available on the market; virtually every subject matter n 
school curricula for every grade level has been developed into courseware (Computers/Associates, 1990). 

However, no matter how good the courseware may be, the efiect of using educational software has been inconclusive. 
One reason or this may be the difference in learners' attitudes toward using computers. As Bloom (1976) contends, 
learning in the affective domain can influence the cognitive learning outcome, and motivation is die key to influence 
learning. The implication of Bkxxn in computer education is that only diose who are motivated and have positive 
attitudes toward computer may learn from computer-based instructions. Those who have negative attitudes toward 
computers may not get the benefit of the effective courseware. 

Addressing the affective factor, therdine, is important in cadet to have the successful curriculum using computers. 
This presentation will review the current research in computer anxiety in relation to math anxiety, test anxiety, and 
Spietburgefs (1966) trait and state anxiety. It also will address die suggestions for inccKporating computer 
curriculum in schools in terms of instructional design and environmental design zspccts. 
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THE EFFECTS OF ADVISEMENT ON ACHIEVEMENT IN 
LEARNER-CONTROLLED INSTRUCTION 



Rowena S. Santiago and James R. Okey, The University of Georgia 

The effectiveness of learner control in computer-based instruction has not been optimized due to learners* difHcuIties 
in making good decisions. Therefore, learner control with advisement has been recommended for CBI. But which 
type of advisement will work for which type of learners remains a {HDblem to instructional designers. 

This paper investigated three types of advisement (ads^ve, evaluative, combined) used in CBI widi learner control 
conditions and their effects when used among learners of varying locus of control (internal, middle, external). A 
randomized, posttest only design was used. Subjects (N=74) worked on a CBI module dealing widi Gage's events of 
instruction and received advisement in the practice phase of the lesson. The dependent variable was posttest 
perfofmancc. 

Main treaunent effects and interaction were tested using a two-way ANOVA« When non*significant interaction and 
significant main effects were found, mean scores were compared using Fisher's FLSD. Results showed signifk:ant 
differences among both the advisement treatments and the LOC groups. Adaptive advisement resulted in better 
perfonnance and internals did better than externals on the posttest Adaptive groq) students were also found to follow 
advisement more frequently and chose to do mote practice than those in the other two groups. Internals and externals 
chose to do similar amounts of practice but externals folk)wed advisement more frequently than internals. 

Rowena S. Santiago 
The University of Georgia 
607 AderhoU Hall 
Athens, Georgia 30602 
(404)542-3810 



CAI MENUS AS LESSON STRUCTURING DEVICES 



Robert L Schuerman and Kyle L Peck, Pennsylvania State University 

There are numerous guidelines for die proper design of menu systems in educational software, but there is little 
empirical evidence to suppcxt these guidelines. A beuer understanding of the role of a menu system as an implicit 
lesson-structuring device in learner control CAI environments would allow designers to anticipate the learner's 
dominant selection patterns and optimize the menu structure accordingly. 

A learner-centered CAI lesson on the classification of questions according to "Bloom's Taxonomy" provided 100 
undergraduate subjects with 15 independent learning activities selected from three pull-down menus, available across 
the menu bar on an Apple Macintosh computer. The study investigated the effects of 1) sequentially numbering the 
menu options, 2) "checking ofT completed menu entries, and 3) a typical, unenhanced (control) condition on the 
subjects* selection patterns. 

Analyses designed to assess the sequentiality of selections (within and across menus), tendencies to complete entire 
menus before moving to the next menu, tendencies to re-enter completed activities, and die degree of deliberation 
during menu selections revealed no significant differences. Subjects, regardless of treatment, tended to proceed 
through menus in a sequential manner. 

Robert L. Schuerman 
Pennsylvania State University 
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Stale College PA 16802 
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THE INFLUENCE OF GRADE MOTIVATION IN A HYPERMEDIA LEARNING 

ENVIRONMENT 



Tom Sextro, John B. Brungardt, and Jack Byars 

The purpose of this study was to detennine the influence of grade moUvation on students in a hypermedia learning 
environment Dependent variables were persistence, as measured by the number of options chosen in learner 
controlled structmed hypermedia lessons, and posttest achievement The results indicated that high grade motivation 
subjects chose signiflcantly mm options than did low grade motivation subjects (p<.01)* No significant difference 
of grade motivation on achievement was noted. Repeated measures testing resulted in a significant interaction effect 
of lesson experience on the number of options chosen for each of two lessons (p<.OS). High grade motivated 
students chose less cations on the second !esson, whereas low grade motivated students chose more options on the 
second lesscxt 
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WAIT TIME AND GUIDANCE IN INTERACTIVE VIDEODISC INSTRUCTION 



Eric E. Smith, University of Northern Colorado 

A series of 5 studies investigating the effects of forced pauses, guidance, and learner control during the presentation 
of uitorial information have been conducted The goal of the research is to begin developing a model of instructional 
variables that can be controlled by the designers/developers of CBT to enhance the effectiveness of the instruction. 

While not conclusive, the results indicate that the use of forced pauses may benefit learners when strategically placed 
after information has been presented and before any review questions or summaries. The use of guided processing 
during those pauses is also supported However, the results indicate that the guided processing used was not effective 
when subjects controlled the pace of guidance screens and the score on the post test was of minimal value. 

The design was essentially the same across all suidies, with the exception of the number of treatments in a particular 
study. The same program was used for all treatments, except for the changes in the specific variables investigated. 

This presentation will cover the theoretical basis for the studies, the speciflc results of each study, and the 
imj^ications for the design and development of interactive videodisc instructional systems. 

Eric E. Smith 

University of Northern Colcxado 

Division of Research Evaluation and Development 

Greeley CO 80639 
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DESIGN-BY-ANALOGY: AN ALTERNATIVE APPROACH 



Steven D, Tripp, University of Kansas 



In his book, Designing Instructional Systems, Romiszowski cites Polya*s approach to problem solving as a 
scientific model for instructional design. Among other things, Polya's method involves the analogical process of 
searching for existent solutions to similar problems. Although Romiszowski mentions this as a possible design 
methodok)gy, he does not work it out in any detail. Romiszowski's model, like all standard ISD models, assumes 
that instructional designers stait from scratch for each project 

The purpose of this paper is first to work out in detail the implications of the Polya model of problem solving, 
second to review literature on analogical reasoning, and then to propose an alternative model of instructional design 
based on the use of analogical reasoning. Justification of the model will be based upon research on the nature of 
anabgical thought and an examination of analogous design models from other fields. 



THE EFFECTS OF VERBAL AND VISUAL METAPHORS ON LEARNING FROM A 
HYPERTEXT BILINGUAL DICTIONARY 



This study is an extension of one presented last year. For this study students were presented with a 120-word 
Japanese-English hypertext dictionary and given 15 minutes to learn as many Japanese words as possible. The 
dictk)nary consisted of 24 semantic clusters of five words each. All subjects received a HyperCard stack with identical 
links and buttons. However, the amount of information available to students was varied between groups. Two 
crossed between-subjects factors were investigated. The first was the presence or absence of a verbal metaphor which 
served as a kind of advance organizer. The second was the presence or absence of a visual metaphor. Dependent 
measures were recognition and recall of vocabulary. We hypothesized that a lack of information about the semantic 
hypcrspace would cause disorientation which would in turn draw upon limited mental resources, reducing learning. In 
the previous study, verbal and visual information did not have an additive effect on retention. We speculated that the 
advance organizer, which was written in literal terms, and the metaphorical graphics induced conflicting mental 
models which reduced achievement. The present experiment is an attempt to test that hypothesis by presenting 
congruent verbal and visual information. 



Steven D. Tripp 

Instructional Technology Center 



School of Education 
University of Kansas 
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Steven Tripp and Warren Roby, University of Kansas 
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A HOLISTIC MODEL FOR INTERACTIVE INSTRUCTIONAL EVALUATION: 

THEORY AND TWO CASES 



Susan A. Tucker and John V. Dempsey, University of South Alabama 

This presentation focuses on designing and implementing effective interactive instructional evaluation (IIE). HE is 
the systematic study of an interactive problem or innovation in order to make effective decisions or recommendations 
about future actions. Specifically, the presentation has three goals. The first goal is to analyze briefly the strengths 
and weaknesses of prevailing evaluation models (e.g., 6bjectivesH)riented, consumerH)rientcd and management- 
oriented) in order to provide a grounding context Second, the IIE model will be presented which addresses 13 major 
theoretical and practical issues faced by professionals woilcing with interactive instruction. Assuming evaluation is 
only as good as the shared reality that the evaluator is able to coalesce across significant players, the authors suggest 
13 critical polemics which capture overt and hidden agendas but do not limit our capability to describe and judge 
program elements. Using a negotiation process to reveal these perceptions, operative values can be embodied in the 
major questions which focus the evaluation paradigm. In addition, these values can serve as judgment criteria for 
decision-making. A final purpose is to describe specific steps, tools and techniques of IIE to compatibly complement 
the design, development and delivery of technologically mediated interactive instruction. Examples will be drawn 
from IIE evaluations applied to two medical education projects, one short-term and the other operating over a three 
year cycle. 

Susan A. Tucker 

DqH (rf Behavioral Studies & 

Educational Technology 
Room 205, ILB Building 
University of South Alabama 
Mobile, Alabama 36688 
(205) 460-601 1 or 460-6201 



A COMPARISON OF TWO CONTENT SEQUENCING THEORIES 
APPLIED TO HYPERTEXT-BASED INSTRUCTION 



Kathleen Kemps Wagner, University cf Northern Colorado 

This presentation discusses the application of Elaboration Theory and Structural Learning Theory to 
hypertext/hypermedia instructional systems. The subjects will be employees of a large electronics manufacturing 
Hrm in Colorado. Results of a study compari.ig learner performance following instructional treatments that differ 
only in the theoretical approach for sequencing '^le material will be discussed and implications for the design of 
hypertext/hypermedia materials jH-esenied. This study will be conducted in September and October 1990. While both 
Elaboration Theory and Structural Learning Theory have been around since the 1970's, little empirical evidence to 
support either theory exists. In addition, both theories suggest sequencing structures for material that have analogs in 
the structure of hypertext documents. The Elaboration Theory analog is a node-linked structure while the Structural 
Learning Theory analog is a hierarchical structure. This study is a first step in the investigation of the application of 
instructional design models to hypertext presentation/instruction systems and, to a limited degree, a validation study 
of the models selected. This presentation reviews the theories, their application via hypertext (including completed 
courseware) and the results to date. 

Kathleen Kemps Wagner 
University of Northern Colorado 
2913 Escondido Drive 
Loveland, Colorado 80538 
(303)669-3469 
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THE PRESENTATION AND ORGANIZATION OF MENU ITEMS 
ACCORDING TO NOVICE AND EXPERT USERS* NEEDS 



Linda Af . Wang, University of Georgia 

This presentation discusses two components, the representation and organization of menu information, and menu 
accessibility for manipulating the interactive computer interface with regard to novice and expert users' needs. 
Guidelines for representing the menu information more effectively through the appropriate interface stj'le will be 
discussed, along with the organization of the menu information in relation to the use of menu sel^don systems. 

Linda M Wang 
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COOPERATIVE LEARNING, INTERACTIVITY, AND YOUNG ADULTS: 
EVALUATING STRATEGIES FOR STUDENTS COLLEGE AGE AND OLDER 



William H. Wheeler, Jr., The University of Georgia; and Linda M. Conn, Valdosta State College 

Although students may benefit from "individualized" instruction, for many, individualized instruction presented via 
a computer may actually be synonymous with "isolated" instruction. There may be a number of benefits for 
learners from well-designed cooperative learning programs, including aiding cognitive processing, relying on each 
other for additional help, and positive affective outcomes . Although many cooperative learning programs have been 
implemented, these have usually involved younger learners. Cooperative learning strategies have not generally been 
implemented and evaluated with students of college age and older. 

This paper evaluates the effectiveness of CAST (Computer Assistance for Science Teachers), an interactive videodisc 
program running on an IBM InfoWindow platform which incorporates a cooperative learning strategy. CAST was 
developed at the University of Georgia under a grant from the National Science Foundation. Students completed the 
program, which includes tutorials, simulations, full-motion video, animation, and color graphics, in groups of three. 
All of the participants in the study were college age or older. Quantitative and qualitative methods were employed! 
Qualitative and quantitative results will be presented, and implications for future research will be discussed 

WilUam H. Wheeler, Jr. 

Dqiartment of Instructional Technology 

607 Aderhold Hall 

The University of Georgia 

Athens, OA 30602 

(404)542-3810 



ERLC 



116 12-S 



SIGTAR 



AN INFORMAL MODEL OF THE HUMAN-COMPUTER INTERFACE: 
PROCESS, ERRORS, AND INTERACTION 



William H. Wheeler, Jr. and Hueyching Janice JiK The University of Georgia 

Advances in the computer tools available to instructional technologists and courseware designers have generated a 
need to integrate knowledge from human factors analysis, user interface design, and instructional design. Easily 
learned authoring systems with powerful interface design controls place the complete range of user interface design 
decisions in the hands of courseware designers whose professional training may not include the discipline of human 
factors analysis and its influence on user interface design. Direct control over screen design, text design, graphics 
display, menu options,and branching exerts a profound influence over the learning context and such measurable 
human factors as: time to learn, speed of performance, user error rate, subjective satisfaction, and retention 
(Shneiderman, 1987). 

Objectives of the p^r include: (1) providing courseware designers with guidance and a framework for decisions 
where knowledge of human factors could be influential; and (2) identifying areas of research that would integrate 
knowledge from human factors analysis and instructional courseware design. We will present a model of the human- 
computer interaction system. The model offers a structure for the examination of a variety of context variables and 
error sources originating in four phases of human/system interaction: (1) computer presentation; (2) user response; 
(3) computer checking; and (4) computer feedback. 

v% illiam H. Wheeler, Jr. 

Department of Instructional Technology 

607AdeiholdHaU 

The University of Georgia 

Athens GA 30602 

(404) 542-3810 



ERLC 



'12S 



Selected Abstracts for the 



Special Interest Group for 
Telecommunications 
(SIGTELE) 



118 

130 



SIGTCLE 



EDUCATIONAL TELECOMMUNICATIONS 
AND THE CHICKEN AND EGG PROBLEM 



Bernie Dodge, San Diego State University 



With many education-oriented BBS's and commercial networks becoming available to teachers and trainers, it is 
important to identify the critical attributes which make such systems succeed or fail. A computer model has been 
developed based on analysis of teachers* use of the FrEdMail network and CSUNet systems in California. The model, 
written in HyperCard, will be used during the presentation to demonstrate various strategies for building a critical 
mass of users. In the model, the central variable is the probability of a user calling the system again, which is 
primarily a function of the cost of accessing the system, the ease of use, and the perceived usefulness of the system. 
The latter variable, in turn, depends on the amount of information on the system provided by other users, and 
information provided by the system operators. The model suggests strategies for increasing use of a 
telecommunications system to get beyond the chickcn-and-egg problem and achi^ ^e a critical mass of the desired type 
of user. The principles underlying the model may be applied to any telecommunications medium which depends at 
least partly on user-supplied information and dialog. 



Training at a distance in both asynchronous and synchronous modes will be addressed. What type of equipment to 
use and how it works will also be featured. 



Bemie Dodge 

Educational Technology Department 
San Diego State University 
San Diego, CA 92182-0311 
619/594-7401 



TELECOMMUNICATIONS FOR TRAINING: A TUTORIAL 



Thomas Downey, Boeing Commercial Airplane Group 



Thomas Downey 

Boeing Commercial Airplane Group 
P.O. Box 3707, MS 2T-79 
SeatUe WA 98124-2207 
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NEW PATHWAYS TO A DEGREE: 
USING TECHNOLOGIES TO OPEN THE COLLEGE 



Stephen C Ehrmann and Michael Strait, AnnenberglCPB Project 

The Annenbcrg/CPB Project will announce the winning projects in its funding program, "New Pathways to a 
Degree: Using Technologies to Open the College." 

Many learners, now part-time students or not enrolled, cou<d work more readily toward a baccalaureate if colleges 
used technologies to offer more accessible, effective academic programs. Each New Pathways project will offer: a) 
courses and services that are highly accessible to working and homebound learners, and others; b) rich resources for 
learning* different from but comparable to opportunities traditionally available for full-time students on campus; c) 
opportunities for frequent conversation and exchange between students and faculty; and d) substantial and coherent 
opportunities to woric toward a degree. The projects each use some combination of available information and 
telecommunications technologies. This session will analyze the submitted proposals, and spotlight several of the 
New Pathways projects. 

Analysis of Proposals: Well over 10% of all colleges and universities in this country participated in a New 
Pathways ]xoposal. Mk:hael Strait will present an analysis of what these proposals signal about which colleges are 
interested in using technologies to extend their programs, and how they are planning to do it. 

Discussion by Project Directors: The remainder of the panel will begin with a brief description of the 
winning projects, and a discussion of how the Annenberg/CPB Project hopes to continue to work with all applicants 
arkl others interested in these issues. The discussion will then shift to three of the funded projects represented on the 
panel. After summarizing their basic strategies, the three project representatives will address several generic 
questions, including: a) the toughest problems each program will face as they begin their work, and b) institutional 
and national policy questions that are raised by these new, more (^n kinds of academic programs. 

Stephen C. Ehrmann 
Annenbcrg/CPB Project 
nil 16thStreetNW 
Washington DC 20036 
(202)955-5273 
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AM EXPLORATION OF SOCIAL AND INTELLECTUAL ••CONNECTEDNESS- 
THROUGH ELECTRONIC MAIL IN HIGHER EDUCATION 



Barbara LGrabowski, Syracuse University: and R, Scott Grabinger, Un *y of Colorado at Denver 

The use of electronic communications is increasing in our society today. The purpose of most of the reported uses is 
to reduce problems caused by large gcogrj^hical distances between colleagues. However, some veiy important effects 
from local uses of electronic mail can also be realized. Kuehn (1988) suggests that electronic mail can extend 
classroom discussions, increase ^^connectedness** of students and faculty, and increase the timeliness of those 
interactions. 

Given that a college community should foster frequent intellectual and social exchanges among the students, Kuchn's 
suggestions have great appeal. At two very diverse universities, one in the east, one in the west, one an 
undergraduate and the other a graduate community, we conducted a survey to assess the social and intellectual 
^^connectedness" of their students. The purpose was to answer four questions: 1) Who is most likely to use electronic 
mail?; 2)For what purposes are users communicating via electronic mail?; 3) What factors influence positive 
perceptions and level of use?; and 4) What are the student's perceptions of the social and intellectual impact of 
electronic mail? Demographics, purpose, and frequency of use data were collected to satisfy the first three questions. 
Using a Likert scale, we tested student's perceptions of electronic mail use. While details of the results will be 
presented and compared between institutions, there were some interesting overall results. Analysis of the 
demographics showed differences in perception between gender, with females being significantly man positive than 
males. In addition, those who used electronic mail for exchanging social information, initiating friendships, and 
alleviating boredom also had a significantly more positive perception of electronic mail in general than those who 
did not use email for those purposes. 
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Kuehn, S. (1988). Discovering All the Available Means for Computer Assisted Instruction: Adapting Available 
University Facilities for he Small to Medium-Sized Course. ED 294 284. 

Barbara L.Grabowski 
Syracuse University 
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College of Education - IDD&E 
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COMPUTER MEDIATED COMMUNICATION: 
A HYPERTEXT APPROACH TO STRUCTURING DISTANCE SEMINARS 



Alexander Romiszowski, Karen Jost, and E-Cheol Chang. Syracuse University 

Several approaches to the consu-uction of content-oreanizational tools that assist participants in maintaining a 
sinicturcd view of a computer-mediated seminar over tinv^ arc described. An experimental hypertext envirxKimcnt has 
been used in several different ways, resulting in varying degrees of effectiveness. This paper presents and discu^^scs 
the results of several experiments. 

Alexander Romiszowski 
Syracuse University 
330 Huntington Hall 
College of Education - IDD&E 
Syracuse NY 13244-2340 
(315)443-3703 
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WILL THE CAMPUS OF THE FUTURE HAVE A CAMPUS? THE IMPACT OF 
TELECOMMUNICATIONS ON RESTRUCTURING 
EDUCATION AND TRAINING 



Carol A. Twigg, Empire State College 

This presentation will do two thing's: 1) inform participants about trends in higher education, workforce 
development, the information explosion and telecommunications, and their effects on restructuring higher education; 
and 2) demonstrate an effective, alternative educational model that incorporates telecommunications technology. 
Telecommunications offers the possibility of thinking about different ways of providing instruction and training in a 
cost effective way. In its twenty-year history, SUNY/Empire State College has developed a highly successful, 
distributed, off-campus program serving 10,000 adults per year. We are now integrating telecommunications 
technology into our academic program. We believe the success of our educational model for serving adults has 
implications for restructuring higher education and training. While institutions and organizations may not adopt the 
complete ESC model, our experience offers valuable ideas for rethinking the structures of higher education and 
training. Participants will ieam: why the campus of the future may not have a campus; how Empire State College's 
off-campus program can provide a m^l for restructuring higher education; how telecommunications technology can 
enhance off-campus learning and assist in overcoming its limitations; and, what issues need to be considered as 
telecommunications usage grows in higher education. 

Carol A. Twigg 
Empire State College 
1 Union Ave 

Sarasota Springs NY 12866 
(518) 587-2100 



A COST EFFECTIVE ALTERNATIVE TO VIDEO TELECONFERENCING 



Linda M. Wang, University of Georgia 

This presentation describes a cost effective distance education course utilizing Macintosh microcomputers, HyperCard 
displays, a conference telephone, and a modem for two-way communications. A graduate statistics course was 
delivered simultaneously at two sites 100 miles apart. The conference telephone carried the professors lecture and 
allowed students at both sites to communicate with the professor and each other. HyperCard stacks supported the 
lecture. 

L'nda M. Wang 

Department of Instructional Technology 

607 Aderhold Hall 

The University of Georgia 

Athens GA 30602 
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NETWORK TECHNOLOGY: ITS IMPLICATIONS TOWARD 
EDUCATION AND ITS IMPLEMENTATION 



Blair R. Bernhardt and Denise LePage, Lehigh University 



Abstract 

With networking ux:hnology being applied lo all levels 
of education, the question becomes one of what 
pedagogical implications does this technology have 
towanl education? E>oes it simply provide a means of 
communication, or does it have broader implications? 
This paper will assess these questions and describe the 
implementation of networking technology at Lehigh 
University. 

Educational Implications 

The typical user of networking technology simply 
wants easy access to computing resources. As Redlaw^ 
(1988) pointed out: "At its simplest level, connectivity 
means access." (p.358). Networking technology 
involves more than simply connecting computers 
together for simple data manipulation. Schambcr (1988) 
described an information network as having the 
characteristics of widespread coverage, a wide variety of 
services, distributed conuol, security, multimedia 
capability, integrated software, adaptability and 
expandability, and reliability. So, with networking 
technology including everything from local area 
networks (LAN's) and digital switches to backbone 
networks with high-speed dau, audio, and video 
capabilities, what are the implications of this 
technology toward education? 

Connectivity 

Educational implications of networking technology 
with respect to connectivity are dependent upon the 
level of networking technology being implemented. 
Levels to be discussed here include local area networks 
(LAN*s), campus-wide networks, and national or 
intenuuional networks. 

In educational settings, local area networks are typically 
found in microcomputer classrooms. LAN's consist of 
individual microcomputers which are interconnected, 
lypicrUy via a printed circuit board plugged into each 
microcomputer, and wiring connected to the circuit 
boards. An integral part of a LAN is the LAN file server 
which is a microcomputer dedicated to running the LAN 
and which contains all of the software for the 
microcomputers on the LAN. As for educational 
implications of LAN's, Thrush and Hardisly (1989) 
state that computer networks [LAN's] may provide for 
interaction between a user and knowledge contained on 
the network, between the user and other users, and, 
through woid processing, between the user and the text. 

Campus-wide networks are typically either backbone 
networks or digital switches (i.e., a digital phone 



system). These types of networks have similar general 
capabilities, but the backbone network is capable of 
much higher data transmission rates and can 
interconnect other networks, such as LAN's. As 
Redlawsk (1988) points out, most users of these types 
of networks just want to get from point '*A** to point 
"B**, connecting one computer or terminal to another 
computer, with a minimum amount of effort The 
educational implications of this type of network are 
dependent upon the distributed accessibility of 
computing resources. For example, the user can connect 
to the library computer and any number of academic 
computers, all from one location. 

Campus-wide networks, especially backbone networks, 
are often connected to external networks. These external 
networks may be national or even international in 
scope. Two of the larger of these networks are CREN 
(Corporation for Research and Educational Networking, 
formerly BITNET), and the IntcmeL Dyrcnfurth (1986) 
describes the capabilities of BITNET as including users 
sharing information via real*time terminal messages and 
elecU'onic mail, transfer of documents and files, and 
access to data services. The Internet is really a collection 
of interconnected networks which provides the same 
basic services as CREN, but at higher speeds and with 
the ability to actually log in and utilize remote 
computer resources (assuming one has an account or 
permi sion to do so). The educational implications of 
these external networks range from simply 
communicating electronically with a colleague at 
another location, to having access to databases and 
information located elsewhere in the world, to having 
ftigh-speed access to computer resources located external 
to one's own institution. 

Pedagogical implications 

The pedagogical implications of networking technology 
can span every subject area. Thrush and Hardisty (1989) 
described how networking technology can be used to 
teach language learning skills. The researchers believed 
that students can develop the ability to communicate 
interactively with information sources, with other 
students, with experts, and with text stored as word 
processing files. Utilizing LAN's, students can acquire 
experience manipulating provided text while practicing 
editing procedures, and sharing text for peer editing and 
collaborative work. 

In addition, Thnish and Hardi^^iy elaborated how students 
can gain experience creating text via '*the progressive 
story" and "the paper cycle" techniques. The progressive 
story involves small groups of students finishing 
several paragraphs which ane started with dictated first 
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sentences. After members of each group check the 
literary characteristics of the resulting stories (grammar, 
oiganization, vocabulary, and creativity), hardcopies of 
the finished stories can be distributed to the entire class. 
The paper cycle is a procedure students and teachers 
follow to progress from the brainstorming of ideas to 
the final draft of a paper. Lastly, Thrush and Hardisty 
suggested conferencing within a structured activity to 
promote interactive communication among students 
who may otherwise feel too self*conscience to speak or 
share ideas in a traditional classroom environment 

Waugh, Miyake, Levin, and Cohen (1988) described 
how nctworicing technology can be used to teach 
problem solving skills. In a particular problem solving 
project involving students from four different countries, 
solutions to a real-worid problem were synthesized from 
students* research. Their descriptions of the problem and 
possible solutions were sent to all participating studeuts 
via a network and then analyzed by all the groups of 
students. After acquiring additional needed research, 
students made decisions on the solutions they found to 
be most feasible for the problem in their respective 
locations. 

Although a similar activity could be implemented in a 
more traditional classroom manner, the network 
technology provided a cooperative educational venture 
which included "a meeting place for like-minded teachers 
and students from all over the worid" (p. 5). As stated 
by Thrush and Hardisty (1989), "development of 
network use is in its infancy, but even now, it provides 
some opportunities for the teacher an^ the learner that 
are almost impossible to achieve otherwise" (p. 4). 

Communications 

Whether it be a group of students utilizing a LAN to 
work together on a class project, or researchers 
collaborating at different institutions, the primary 
educational implications of networking technology 
relate to communications. This is not to imply that 
networking technology applies strictly to electronic 
communications. Studies such as that conducted by 
Daly, Weber, Vangelisti, Neel, and Maxwell (1987) 
show that networking technology can be used as an 
effective tool to not only explore how people 
communicate, but also to explore the thoughts people 
have as they communicate. By documenting the 
conversations individuals had over the networked 
microcomputers and the thoughts that were verbalized 
orally during the respective netwoiked communications, 
the researchers observed three general thought processes 
taking place although not necessarily evident in the 
microcomputer conversation; inferencing, planning and 
coping with violations of conversation maxims. 

Implementation at Lehigh 

At Lehigh University, networking technology has been 
implemented in a number of different ways including 
local area networks (LAN's) within microcomputer labs. 



networics of workstations, a campus-wide digital phone 
system (digital switch), and a campus-wide, fiber optic- 
based backbone network. The backbone network 
provides connectivity between all of the other campus 
networks as well as national and international 
connections. 

inteCom and the Network Serve*^ 

At Lehigh, most people refer to the University's 
InteConi digital phone system as the "campus 
netwwk." The InteCom system provides simultaneous 
voice and data communications to the university offices, 
labs, most classrooms, and most dormitory rooms. A 
menu-driven microcomputer-based program was 
developed at Lehigh to dial the phone and establish 
connections to LehigL University Computing Center 
(LUCQ academic mainframe computers, to the on-line 
catalog system of the university library system, known 
as ASA (Automated System Access), as well as to other 
dq>artmental and administrative computers. 

One of the mainframe computers which can be accessed 
through the InteCom system is an IBM 4381, which 
has acquired the nickname of the "Network Server." The 
Network Server provides information and 
communications services to users of the InteCom 
system. On this system, course assignments can be 
submitted and returned via electronic mail. Course- 
oriented bulletin boards and conferences provide a level 
of interaction between faculty members and students 
which ^ould not be present in non-networked 
environments; students who may hesitate tO ask 
questions in class can ask questions electronically; 
faculty can post answers to questions, new assignments, 
and helpful class information. This system also allows 
for easy distribution of faculty written software as well 
as the filling out of on-line forms for requests for 
information or campus services. 

Local Area Networks 

Microcomputers at LUCC microcomputing sites arc 
interconnected by means of local area networics (LAN's). 
In addition to the connection to the LAN, each 
microcomputer is also connected to the InteCom system 
through a serial port This allows each microcomputer 
to dial and communicate with other computers such as 
mainframe computers. In addition to terminal 
emulation, this connection allows files to be uploaded 
and downloaded to and from the mainframe, 
respectively. One of the functions of the LAN is to 
provide software for this connection. 

Educationally, these LAN's not only provide students 
broader exposure to software than would be available in 
a non-LANed environment, but also provide a medium 
for student/faculty interaction. Holding classes in a 
LANed classroom allows instant distribution of files 
and data, as well as instant elecu^onic sharing of ideas. 
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The Backbond Network 

In addition to the InteCom system and local area 
networics, Lehigh University is in the process of 
installing a TCP/IP-based (Transmission Control 
Protocol/Internet Protocol) backbone network between 
various campus buildings. The backbone network is a 
separate, high-speed network which links other 
networks. Currently, internal LUCC Ethernet networks 
(including LUCC staff LAN's) arc connected to the 
brxkbone, as are some departmental LAN's within the. 
connected buildings. It is anticipated that, eventually, 
all LUCC site LAN's will be interconnected by means 
of the backbone networic 

The backbone network provides users of LAN's 
connected to it with the means to send and receive data 
at speeds far beyond those supported by the InteCom 
system. Since the backbone network is connected to the 
LUCC machine room Ethernet network, which in turn 
is connected to external networks, all users connected to 
the backbone are also provided access to the various 
services of the external nctwc^. 

Electronic Mall via CREN and the Internet 

The University is connected to tvo different types of 
external networks; CREN (formerly BITNET, via the 
IBM 4381 "Nelworic Server"), and the Internet (via all 
backbone network hosts). Gateways between CREN and 
the Internet provide users of one of these networks the 
ability to send electronic mail to users of the other of 
these networks. 

At Lehigh, CREN is strictly for electronic mail. All 
users of the Network Server, as wrll as all users of 
LUCC's CYBER 850 and VAX 8530 mainframes 
(which are Internet nodes), can send to, or receive 
electronic mail from, all CREN and Internet users. 

Internet Communications 

The Internet provides services in addition to electronic 
mail by way of application programs such as TEI-NET 
and FTP (File Transfer Protocol). TELNET allows 
users to log in to remote computers from local 
computers, while FTP is for transferring files between 
computers that are connected to the Internet 

Since both the CYBER 850 and the VAX 8530 are 
connected to the Internet, FTP can be utilized as a quick 
and effective method for transferring files between diesc 
LUCC machines. In addition, TELNET can be utilized 
from either of these machines in order to access the 
other without logging out of the first machine. This can 
be an effective tool in the development of projects 
which require the use of both the CYBER 850 and the 
VAX 8530. The ASA on-line catalog system can also 
be accessed from either of these machines through 
TELNET, as can machines external to the University. 
External connections allow collaboration between 



researchers at Lehigh and those at other institutions, 
beyond that provided by electronic mail. External 
connections also allow access to databases and other 
infomaiion "available on the Internet 

Conclusion 

All levels of networking technology have implications 
toward education. Local area networks {Hovide a medium 
for communications and sharing of information between 
people in close proximity, such as members of a class 
v/oticing in a microcomputer classroom. Campus- wide 
networks provide similar capabilities dcross the campus 
and also provide a means through which distributed 
resources can be shared. And finally, national and 
international networks provide a means for sharing 
information and resources far beyond the confines of the 
campus. 

Although still in the formative stage, implications of 
networking technology toward education currently 
include specific learning applications such as language 
skills, problem-solving skills, and collaborative 
learning skills. As networking technology evolves, and 
audio and video capabilities become an even more 
integral part of it, increased opportunities to enhance the 
learning environment will become evident 
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COMPUTER-BASED TRAINING AMONG TOP AMERICAN BANKS 



Pamela S. Nolf» Automattd Financial Systems: and Harold F. Rahmlow; Rahmlow Associates 



Abstract 

This research studied the use of Computer-Based 
Training (CBT) by major American banking 
institutions. The study is a replication of one done in 
the insurance industry and was made up of the top ISO 
commercial and 100 consumer banks. The CBT areas 
that the questionnaire focused on were: usage, 
administration, target audience, hardware and software, 
and types of course content 

Introduction 

The rapid increase of CBT usage by major American 
corporations has been a phenomena in recent years. In 
1988 a study was conducted of the use of CBT in the 
insurance industry (Rahmlow and Marsh, 1989). 
Seeking to extend the research in insurance into the 
banking industry, Nolf replicated the Rahmlow and 
Marsh work in a study among major American banks. 
The current study was conducted in the spring of 1989. 
The population for the study was niade up of the top 
150 commercial banks and 100 consumer banks as 
determined by assets. 

Purpose of the Study 

Corporations arc spending more and more money on 
CBT as a training tool. Recent surveys (Lee, 1987; 
Spicgclman, 1987) indicate that approximately 53 % of 
American corporations, ranging froT. companies with 
50 or more employees to the Fortune 500, use CBT. 
Mote importantly, these surveys show that many of the 
remaining companies plan to begin using CBT in the 
next few years. 

How extensively are American corporations using CBT? 
In 1986, one survey (Ralphs and Stephen, 1986) of 
Human Resource Development (HRD) practices within 
the Fortune 500 companies leported that approximately 
44 % of the responding companies are using some form 
of CBT. The results are somewhat misleading; because 
of the way the research question was worded, the extent 
to which CBT was being used compared to other 
training methods could not be determined. 

The same uudy (Stephen, Mills, Pace, Ralphs, 1988) 
repealed two years later showed that only 17 % of the 
companic,^ reported a high use of CBT. Reported usage 
of all technological advances in training, such as 
interactive video and satellite TV networks, huddled at 
the "almost never used" end of the scale. As the authors 
of the survey concluded, "In terms of significant use of 
contemporary technological advances in the field of 
training and development, these results arc dismal" (p. 
30). 



The previous comment reveals the current 
conceptualization of CBT in the corporate environment 
as a "technological tool." The effect of CBT is to train 
employees to perform a function or task, the emphasis 
being to make training more efTicient and cost effective. 
Therefore, the purpose of this study is to examine the 
following areas of CBT delivery among major American 
banks: 

1. Degree of usage 

2. Administration 

3. Audience 

4. Hardware and software 

5. Types of courses 

By sharing this data, the American banks can make 
more effective use of their training dollars. 

Significance of the Study 

American industries spent an estimated $30 billion a 
year on training. The amount sounds impressive, but as 
Peters (1987, p. 21), author of In Search of Excellence, 
points out, that is small change compared to the $400 
billion spent on hardware. As he challenges, "Wc may 
have gotten away with our neglect of human capital 
development in days gone by, when cheap energy and 
the logic of mass production created operations that 
relied on narrow, specialized labor. But that world is 
long gone." The complex technology required for more 
efficient production needs a more educated work force 
which calls for better training. 

A training methodology increasingly mentioned as a 
solution to this problem is CBT. However, any 
corporation wishing to utilize CBT in an educational 
and cost effective manner must recognize both its 
advantages and limitations. The benefits of CBT are 
mostly discussed in a technological or economic sense, 
while its most serious limitation is in the area of 
matching learner characteristics, such as a preference for 
individual or group instruction. 

Development of CBT is expensive. It can take from 100 
to 300 (or more) hours to produce 1 hour of CBT. As 
G. Gcry (1986) points out, the more complex the 
courseware, the greater the development time. The 
average development cost per hour of CBT is 
$6,000.00; interactive video, which uses video and 
computers to deliver training, costs $12,000.00 per 
hour to develop (Diamond, 1^86). Compared to 
classroom training, however, delivery costs for CBT arc 
low. Scldcn and Schultz (1982) estimate these costs for 
classroom training are about 40 times more expensive 
than CBT over a 4-year period. 
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CBT does offer certain unique advantages in 
instructional delivery.. Many authors (Snyder, 1987; 
Wehr, 1988) note such benefits as availability, 
interaction, self-paced student control, reinforcement, 
and realistic simulations. 

The biggest disadvantage of CBT is that it may not 
match the preferred learning style of many trainees. 
Doiy and Doty (1964) found that the students who 
benefit most from programmed learning have relatively 
low social needs and high academic ability. Dunn and 
Dunn (1978) suggest that programmed learning is more 
effective for highly motivated, responsible students who 
are visually motivated and do not woric well in groups. 
Hopmeir's research (1981) indicates that introverts, who 
are task- and detail-oriented, do better with CBT courses 
than extroverts or perceptive learners. 

The key to successful and effective CBT, as in any 
training media, is still instructional design, which takes 
into account the preferred learning style of the trainee. 
As Godbey (1978, p. II) points out, "Good leaching is 
done by thinlcing first of how it affects the learner, not 
the teacher." J. Woolsoncroft (Snyder, 1987, p. 41) adds 
thai "a CBT course, even if the content is good, can be 
useless unless it's got a good design to it" 

Knowles (1980, p. 59) warns against "panacea- 
addiction** or "a compulsion for neat, simple solutions 
to complex problems." CBT may seem like the perfect 
marriage of technology to instructional design. 
However, unless all aspects of instructional design, 
including learner characteristics, are taken into account, 
the results may not be a perfect union but a messy 
divorce. 

Limitations of the Study 

Since the study involves a broad survey of many 
different banks, the following limitations should be 
considered: 

1. The reason CBT was selected by the bank as 
opposed to some other training medium was 
not considered. More extensive surveys will 
have to provide comparison data with other 
training methodologies. 

2. Evaluation of the effectiveness of CBT is 
considered beyond the scope of this surveyH. 
owever, any consideration of training 
methodology must ultimately address this 
factor. 

3. The number of respondents may limit the 
representativeness of the survey. Since the 
survey is limited to only large banks, the 
results may not necessarily apply to mid- and 
small-size banks. 

Even taking these limitations into account, this survey 
can produce information that helps banks improve their 
delivery of CBT. 



Review of Literature 

CBT usage by American corporations is increasing. It 
is, however, a relatively new training methodology. In 
recent years, most American corporations have been 
faced with rapidly changing economic conditions and the 
challenge of deregulation. In many industries, the 
Federal government has reduced or eliminated laws 
governing pricing and market areas. To respond to the 
increased competition, corporations have had to develc^ 
new products, improve customer service, and lower or 
control prices. 

Technology, such as new software and improved 
hardware, is seen as a vehicle for meeting these 
challenges. CBT is viewed as a training technology that 
can provide large-scale and efficient training. However, 
CBT is relatively new, a product of the very technology 
that it makes more manageable. To understand the 
current status of CBT, it is necessary to understand how 
it developed and became prevalent in the corporate 
environment 

Use of CBT in the Corporate Environment. 

CBT is often spoken of as a training technology in its 
teenage years. For a sixties child, it is now in 
adolescence and still seeking direction. A. Packer (1988, 
p. 10), a researcher at the Hudson Institute, predicts that 
in the next decade, instructional delivery systems may 
pass data processing as the most common function of 
computers in American companies, a change dictated by 
the evolving skill requirements of a developing global 
economy. 

Corporations began to use CBT as a training tool in the 
1960s, although its early applications were limited. As 
early as 1976, IBM used CBT for internal training. At 
the same time. Control Data began to develop PLATO, 
an authoring and delivery system. As P. Holste (1988, 
p. 6) points out, "But it wasn't until this decade that the 
<F14MI>n€€d<F155D> for training on a^:d with 
computers became significant enough to fuel creative 
innovation and broad market growth." 

In the 1980s, CBT exploded onto the corporate training 
scene. As late as 1984, 50 % of the trainers surveyed by 
the magazine Training (Trends, 1985) had hiilr or no 
experience with CBT. By 1989, the annual survey 
showed that 43.3 % (up from 37.9 % in 1988) of 
United States organizations with 100 or more 
employees use CBT. Most companies use CBT to teach 
computer*related applications; however, an increasing 
number are using CBT to teach technical, management, 
and interpersonal skills. 

Hirschbuhl (1987) recently completed a comprehensive 
survey of CBT among eight major American industries 
to determine the level of usage, development 
commitment, and user satisfaction. Some of his 
fmdings are: 

• CBT is most frequently used for data 
processing training (20 to 30 %) 



ERLC 



129 



1^' 



Nolfand Rahmlow 



Vendor-supplied generic courseware accounts 
for the vast majority (70 % in 1987) of CBT 
courses used by corporations. Internal 
development represents only 6 % of CBT 
usage and custom CBT makes up the rest. 
CBT use is division- or department-based, not 
centralized 

Hirschbuhl's summary of the commitment to use CBT 
for in-house training in 1987 among various industries 
is shown in Figure I. E>cregulated industries such as 
insurance, banking, and airlines have shown the greatest 
commitment to CBT; accounting firms used CBT the 
least 

Rahmlow and Marsh (1989) conducted an in-depth 
survey of CBT usage in the insurance industry. Fifty 
nine percent of the responding insurance companies 
report that they are using CBT. In 1978. a similar 
survey conducted by The American College Company 
Coordinators reported only 6 out of 400 insurance 
companies used CBT delivered on a mainframe system, 
the only choice since personal computers were 
extremely rare. Although CBT is only a small pan of 
the total training effort, a significant increase in the use 
of CBT has occurred in the past ten years. 

Since insurance companies and banks are financial 
institutions, much useful data can be compiled by 
replicating Rahmlow and March's survey in the banking 
industry. Are most banks using CBT? Are major banks, 
like insurance companies, using micro delivery more 
than mainframe delivery? Is the training department 
most often responsible for the administration of CBT? 
By answering these questions, both industries can 
benefit from their differing experiences. 

The next topic presents an overview of CBT use by 
banking institutions. 

CBT Use By Banks. Both the banking and 
insurance industries have been greatly affected by 
der^lation and intense competition. Banks have turned 
to CBT because its unique characteristics enable them to 
deliver training efficiently to large numbers of people. 
Banks also use CBT because of the significant cost 
savings particularly when compared to instructor-led 
courses. Zimmermann (1988) investigated use of CBT 
and interactive video by banks and found that the major 
reasons for using them were cost savings, convenience, 
consistency, reduced travel expenses, and decreased 
learning time. 

When a bank is converting to a new software 
application, time is money. In 1985, Bank of 
Oklahoma converted nine recently acquired banks to a 
new deposit system. The bank used CBT to provide the 
training. As S. LaCross (CBT Programs, 1985, p. 
110), conversion analyst, comments, "With CBT, we 
saved time. We also couldn't have covered [the material] 
as well or as consistently." 



An increased emphasis on sales due to deregulation has 
forced banks to produce training about products such as 
types of savings accounts and special loan financing. 
The North American Finance Group of Citibank (J. 
Borger, 1988) uses CBT to deliver data about 
international products and marketing strategies. 
Compared to the previous year, usage by the 600 
relationship managers, who represent the bank to 
institution customers, has increased by 20 % in 1986 
and by 32 % in 1987. Sovran Financial Corporation 
(Pollock, 1989) is using CBT to teach a computerized 
retail sales program. By becoming more comfortable 
with the product, platform personnel can increase sales 
activity. 

Banks are becoming more automated. As Hardwick 
(Zimmerman, 1988, p. 84), president of Electronic 
Learning Systems, says, "With so much information at 
the fingertips of the people who can change the 
business the tellers and platform personnel banks have 
to teach tliem how to use the systcms effectively." Ward 
(1985, p. 3), a member of the Training and 
Development Department of First Union Bank, says, 
"With computer-based training, it will be easier for us 
to train employees to use the corporation's automated 
systems. Learning will be easier because the training 
will take place on the same equipment they use day-to* 
day....We cannot bring hundreds of branch employees 
into the corporate headquarters every time there is a new 
product or (Hocedure to leach them." CBT was developed 
to fulfill the training needs of the expanding 
corporation. 

Tumover is also a problem at banks necessitating repeat 
training at irregular intervals. As Roden (1988, p. 11) 
explains, "The tumover is phenomenal; some banks 
have 100 percent tumover among their tellers." 

In short, CBT represents a training methodology able to 
deal with new technology in the work place, rapidly 
changing economic conditions demanding new banking 
products, and tumover from a changing work force. 

Methodology 

This study describes the methodology used to conduct 
the study. Topics are research questions, sample, 
instrumentation, procedures, and summary. 

Research Questions. The following areas were 
surveyed: 

1. Usage 

2. Administration 

3. Audience 

3 . Hardware and software 

4. Types of courses 

The following section lists and discusses each of these 
major research topics. On the actual questionnaire there 
may be several items addressing each of the above 
topics. 
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1 . Is your Company ciurcnily using any form of 
computer-based education? The primary 
research topic sought to determine whether 
major banks were using CBT. If a bank was 
not using CBT, the questionnaire asked if use 
was anticipated within the next year and the 
type of system to be purchased. 

2. Which Department has the responsibility for 
the administration of this effort? The second 
research topic established the department(s) that 
administer CBT. Individual departments, such 
as commercial lending or installment lending, 
may administer CBT within their own groups 
or CBT may be under the jurisdiction of an 
education or training department. Joint 
departments may also be responsible for its 
administration. Within larger financial 
institutions, subsidiary banks or even branches 
may control their own training. 

3. Which of the following groups use courses 
you offer? corporate management, branch 
management, branch support staff, loan 
officers, loan support staff, q)erations officers, 
operations support, trust officers, other. The 
third research topic determined which groups 
within the bank were using CBT. For 
example, it may be used primarily for data 
piocessing or loan officer training 

4 . What type of microcomputer hardware are you 
using? Docs your organization use modems 
with PC*s? Does your organization use a LAN 
(Local Area Network)? What type of 
instructional system are you using? The fourth 
topic concerned the types of hardware and 
software used to deliver CBTCBT can be 
presented via a mainframe computer, a personal 
computer, or personal computers configured in 
a network. Numerous presentation systems are 
available for the range of hardware types. 

5. How many hours of training are currently on 
your system? Please attach a list of courses. 
The last question concerned the types of 
courses offered. Respondents were asked to 
estimate the total hours of available CBT, 
provide a list of course subjects, and indicate 
how they were obtained developed in-house, 
purchased from outside vendors, or developed 
by consultants to the bank^s specifications. 

Sample 

The population was made up of the top ISO commercial 
banks (finance business and commerce loans) and 100 
consumer banks (finance loans to individuals for 
personal, family, or household reasons) as determined 
by assets. This listing was provided by the October 
1988 issue of American Banker. 

Current Federal regulations permit American banks to 
perform both commercial and consumer functions. 
Therefore, there was an overlap in the sample and some 



banks were in both categories, as shown in Figure 2. 
As a result, only 165 questionnaires needed to be sent.. 

Since the population was relatively small, 
questionnaires were mailed to the human resource 
directors of the entire population. Names were obtained 
from the 1988 North American edition of Polk's Bank 
Directory. 

Instrumentation. A questionnaire, shown in 
Appendix A, was developed to address the research 
topics. In some cases, more than one item in the 
questionnaire dealt with a specific research question. 
This questionnaire was adapted from one used by H. 
Rahmlow and N. Marsh (1989) in their survey of CBT 
usage in the insurance industry and auempted to extend 
their original research. 

The validity of the questionnaire was tested by 
submitting sample questionnaires to three experts, 
directors or managers of CBT at a bank. They were 
asked to analyze the content validity, or how well the 
questionnaire items measured the intended content area. 
They detennined if the proper areas were surveyed and 
that there was a suitable number of questions to 
properly survey each topic. These experts also examined 
the face validity, or degree to which tiie questionnaire 
measured what it was intended to measure. The format 
and wording was revised based on their comments. 

Reliability, whether the test consistently measures the 
same thing over a period of time, was not specifically 
investigated since this particular questionnaire was to be 
administered only one time. 

Procedure. In February 19£. , three selected human 
resource directors or their equivalent in CBT training 
were contacted by telephone to determine if they were 
willing to participate in the development of a survey on 
CBT in the banking industry. All three agreed to take 
part in the research. They were mailed surveys and asked 
to complete it and analyze its design. Based on their 
comments during follow-up calls, the design was 
modified. Their responses were used in the final 
tabulation of survey results. 

On March 15, 1989, the human resource directors of the 
top banks in the United States were mailed 
questionnaires. A cover letter described the study and 
piromised confidentiality. Tlie participants were asked to 
return the questionnaire by March 31, 1989. 

Also included was a letter from S. Switzer, President of 
the Philadelphia-Delaware Valley Chapter of the 
American Society for Training and Development 
(ASTD). This leuer stated that the results of the survey 
were to be shared with trainers in the banking industry 
and other members of ASTD. It asked the respondent to 
participate in this consolidation of data on a new 
U^ining technology. 
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Survey Results 

This section summarizes the results from the 67 
surveys returned. It analyzes the survey data to 
determine if the strategies used to develop CBT are 
reflected in its delivery by major American banks. The 
following areas are examined: degree of usage, 
administration, audience, hardware and software, and 
courses. It also discusses the response level and presents 
a study sumniary. 

Degree of Usage. CBT usage among major banks is 
prevalent since 51 banks (77 %) use some sort of CBT. 
An additional 5 banks anticipate beginning to use CBT 
before the end of 1989. Several respondents indicated 
through marginal notes that the use of CBT is new and 
2 banks noted that they are scaling down their CBT 
development 

Administration. As shown in Figure 3, the 
education and/or training department is responsible for 
all or part of the administration of CBT at 52 % of the 
banks. At 21 % of the banks, CBT is administered by 
data services. Joint responsibility between at least 2 
departments is listed by 15 % of the banks. The 
remaining banks report that CBT was administered by a 
specific depaitment such as commercial lending or trust 
No historical data is available to determine if this trend 
of responsibility has been shifting over the years. 

Audience. The audience, or job classifications, for 
CBT delivery docs reflect a bias as shown in Figure 4. 
The highest use of CBT is reported in the operations 
area which is responsible for 30 %. The next highest 
usage is by loan officers and support staff (24 %). 
Branch management and support staff account for 23 % 
of course usage. The remaining course usage is other 
(12 %), trust officers (6 %), and corporate management 
(5%). 

The above figures categorize CBT usage by department 
The same numbers examined by job classifications, 
regardless of department, show that training is about 
evenly divided between management (46 %) and support 
staff (42 %). Thus all levels of bank personnel are being 
uained by CBT. Some banks even indicated that 
everyone in the bank used some of the CBT courses. 

Hardware and Software. The hardware and software 
results are as follows. Among those banks using 
mainframe course presentation, 47 % use Goal's 
Phoenix. Two of the banks anticipating using CBT 
before the end of the year also will use Phoenix. Only 
11 % of the banks use IBM HAS or HPS as a 
mainframe presentation system. Since the other 
presentation systems have only one or two banks using 
them. Phoenix is clearly in the lead in the mainframe 
environment. 

The micro computer hardware most often used is IBM 
(84 %). About 58 % of the banks use modems and 45 
% report using Local Area Networks (LANs). However, 



marginal notes indicate that micro usage is hard to 
measure. For example, tutorials for micro applications 
are often distributed on floppies so that student usage 
cannot be tracked easily. Although the use of CBT is 
high at 77%, use of interactive video is limited at 14%. 

Courses. Most of the CBT courses involve the 
computer or computer applications. A large number of 
courses involve learning programming languages or 
how to use the computer. Some courses are about using 
new software applications such as commercial lending. 
Other courses, such as Regulation C, involve changing 
government regulations in banking. A few courses 
concern improving professional skills such as writing. 
Even though course content cannot be specifically 
determined from the title alone, the trend for using CBT 
courses to present training on new technology or 
government regulations is quite clear. Not all banks 
provided lists of courses. The most frequently listed 
courses and the number of banks rqx>rting them are: 

Computer Software Courses 

Focus 4 

DOS 3 

JCL 3 

CMS 2 

Easytrieve 2 

SAS 2 

TSO 2 

VTAM 2 
Computer Hardware Courses 

Terminals 4 
Computer Literacy Courses 

Bases 4 

Data Processing 2 

Electronic Mail 2 
Banking Application Courses 

Commercial Lending 6 

Customer Information Service 4 

Branch Authoring 3 

Check Processing 3 

Deposits 3 

Customer Information File 2 

Safe Box 2 

Trust 2 
Banking Terminology Courses 

Products 5 

Currency Transactions 4 

Individiial Retirement Account 4 

Regulation C 3 

Platform 3 
CBT Related Courses 

How to Use CBT 3 
Professional Skills Courses 

Writing Skills 5 

Lotus 3 

Performance Improvement 2 

Stress Management 2 

Time Management 2 

Typing Skills 2 
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The number of hours of computer-based instructional 
courseware available at each site varies widely. The 
range goes from 0 (the bank was developing its first 
course) to 300 hours. The distribution is skewed to ihe 
low end with over half the banks having under 22 
hours. This data is hard to interpret because many 
respondents had trouble determining this factor since a 
unit making up a CBT course may take from S to 4S 
minutes. 

Most banks are obtaining their CBT courseware from 
outside vendors (53 %) or contracted developers (23 %). 
The banks who develop their own CBT (49 %) do so in 
a centralized (76 %) as opposed to a decentralized 
environment (24 %). Note that the percentages for 
obtaining coiu^ware total more than 100 % since most 
banks use a combination of methods. 

The majority of banks developing their own courses (S9 
%) are authoring only on the mainframe; 20 % are 
authoring in both mainframe and micro environments; 
and 9 % are authoring only in a micro environment. 
The remaining 12 % do not give a clear indication of 
how they are authoring. 

Phoenix is the overwhelming choice for an authoring 
language (74 %). Every other language (Boeing 
Scholar/Teach. IBM IIAS/IIAP. SAM. etc.) is 
mentioned by only one or two banks. 

Costs of authoring CBT are allocated as follows: 
Training or CBT department. 31%; No formal system. 
19%; Other. 19%; Combination of methods. 15%; 
Sponsoring department. 13%; and Training and sponsor. 
3% 

Student usage is measured by the numbers of students 
who use CBT and the numbers of hours they spend on 
CBT. In the mainframe environment, banks report an 
average per month of 594 student hours for 286 
students. In the micro environment, banks list a much 
lower average per month of 4 1 student hours for 24 
students. 

In conclusion, banks are offering CBT courses mainly 
about computers or computer applications, not about 
conceptual subjects such as problem solving skills or 
motivation. Most of the courses are obtained from 
outside vendors. The hours of use by students vary 
tremendously from bank to bank, but most of the 
recorded use is in a mainframe environment 

Response Level. The response rate was 41 % since 
165 surveys were mailed and 67 relumed. One of the 
surveys was unusabIe(The return indicated that CBT 
usage, if present, was controlled by the affiliate banks. 
Subsequent mailings to these affiliates produced no 
response.) This was a representative sample because the 
returns were evenly disuibuted by bank size and type. 
Because of this representation, it was decided not to 
perform a second mailing. 



Results Summary. CBT is a training technology 
used by 77% of the responding banks. Most of the 
banks use CBT to train individuals in procedures such 
as programming or content specific areas such 
government regulations. The highest use of CBT is in 
the operations area (30 %) followed by loan ofHcers and 
support staff (24 %). CBT is used to train both 
management and support staff. In general, the education 
and/or training department is responsible for all or part 
of the administration of CBT at 52 % of the banks; 
however, the data processing area administers CBT at 
21% of the banks. 

Conclusions 

The study has documented the status of CBT among 
major American banks at the close of the decade. CBT 
is a training technology of growing importance. Given 
the relative infancy of the technology and the great 
signiflcance of training in the banking industry, it will 
be important to further monitor the development of this 
technology. 
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THE AT-RISK ADULT LEARNER: DEFINITION AND SOLUTIONS 
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Abstract 

There is a serious lack of provision of educational 
opportunities for AT-RISK adults. Many underlying 
problems relate to this lack of opportunity; four will be 
discussed. l)the definition of an AT-RISK adult is not 
clear, 2)the need for lifelong learning and education 
opportunities for the AT-RISK adult learner remains 
unseen, 3) creating an environment for this population 
is a complex task, and 4) technological advances in 
education are not being utilized with the AT-RISK 
adults. 

Determining solutions for the lack of provision of 
educational opportunities for the AT-RISK adult is 
complex. Four solutions will be described: 1) review 
and compare methodologies utilized in countries around 
the world, 2) consider instructional design as an 
alternative research paradigm, 3) examine the 
possibilities of Outdoor and Computer Education for 
Adult Needs Ranch, and 4) examine the possibilities of 
using «ui affective domain instructional model within 
the Outdoor Computer and Education for Adult Needs 
Ranch. 

There is a serious lack of provision of educational 
opportunities for AT-RISK adults. Many underlying 
problems relate to this ^ck of opportunity; four will be 
discussed. First, the definition of an AT-RISK adult is 
unclear. Second, normal adults are only beginning to 
see the need for lifelong learning and education 
opportunities, therefore the AT-RISK adult learner 
continues to remain unseen. Third, creating an 
environment for this population is a complex task. 
Finally, technological advances in educating the special 
needs population have only been utilized with children 
and with a select group of adults. 

Ai*Rlsk Definition 

How can the AT-RISK adult be defined? One method, 
although somewhat cold, is to identify and classify the 
members of this population. AT-RISK depicts a 
potential or minimal amount of potential for change. 
Three population groups are described as having a 
potential problem with learning/educational change. 

Children with Developmental Disabilities are often 
given an appropriate education until age 18. When they 
reach adulthood their educational opportunities become 
minimal. Group one is identified as the adult 
Developmentally Delayed 

A second group of AT-RISK adults arc the functionally 
illiterate. These people have successfully managed to 
remain employed, maintain relationships, and often 



raise children. Literacy groups are only beginning to 
develop educational programs to meet the needs of these 
adults. 

The third group of AT-RISK adults are those with 
psychological problems. Reasons for these problems 
may vary, however they have a serious effect on 
learning. Often the psychological issue may be 
addressed but not in relationship to learning. 

There may be additional groups within the AT-RISK 
definition. However small and simple, these three 
groups are the beginning of an understanding of this 
population and its educational needs. 

Normal Adults/Llfe-Long Learning 

The second problem in providing educational 
opportunities for the AT-RISK adult is directly related 
to how normal adults are just beginning to recognize 
the need for lifelong learning. Five examples of 
contemporary adult learning theory will be highlighted 
to offer better understanding of the foundations of adult 
learning. 

Carl Jung advanced a holistic conception of human 
consciousness, inutxlucing the notion that it possesses 
four functions; sensation, thought, emotion, and 
intuition. His plea for the development and utilization 
of all four functions in balance laid the groundwork for 
the concepts of the balanced personality and the balanced 
curriculum. 

Erik Erikson provided the "eight ages of man," the 
last three occurring during th"* adult years, as a 
framework for understanding the stages of personality 
development: 

1. Oral-sensory, in which the basic issue is trust 
vs. mistrust. 

2. Muscular-anal, in which the basic issue is 
autonomy vs. shame. 

3. Locomotion-genital, in which the basic issue 
is initiative vs. guilt. 

4. Latency, in which the basic issue is indusuy 
vs. inferiority. 

5. Puberty and adolescence, in which the basic 
issue is identity vs. role confusion. 

6. Young adulthood, in which the basic issue is 
intimacy vs. isolation. 

7. Adulthood, in which the basic issue is 
generativity vs. stagnation. 

8. The final stage, in which the basic issue is 
integrity vs. despair. 

In fact, the central role of self-concept in human 
development and learning received increasing 
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reinforcement from the entire field of psychiatry as it 
moved away from the medical model toward an 
cducatiofud model in its research and practice. 

Maslow placed special emphasis on the role of change, 
as becomes clear in the following formulation of these 
elements in the growth process: 

1 . The healthy sponlancous person reaches out to 
the environment in wonder and interest and 
expresses whatever skills they have. 

2. To the extent that he/she \s not crippled by 
fear, to the extent that he/she feels safe enough 
to dare. 

3. In this process, that which gives him t!ie 
delight-experience is fortuitously encountered, 
or it is offered to him by helpers. 

4. He/she must be safe and self accepting enough 
to be able to choose and prefer these delights, 
instead of being frightened by them. 

5. If he/she can choose these experiences which 
are validated by the experience of delight, then 
he/she can return to the experience and repeat 
it. 

6. At this point, he/she shows the tendency to go 
into richer, more complex experiences and 
accomplishments in the same sector. 

7. Such experiences not only mean moving on, 
but also have a feedback effect on the Self, in 
the feeling of certainty, capability, mastery, 
self-esteem, and sclf-tnist 

8. In this ending series of choices of which life 
consists, the choice may generally be 
schematized as between safety and growth. 

9. In order to be able to choose in accord with 
his/her own nature and to develop it, the 
individual must be permitted to retain the 
subjective experiences of delight and boredom, 
as the criteria of the correct choice. 

10. If the choice is really a free one, then we may 
expect him/her ordinarily to choose 
progression forward. 

21. The evidence indicates that what delights the 
healthy person is also "best" for him/her in 
terms of far goals as perceivable by a spectator. 

12. In this process the environment is important in 
many ways, even though the ultimate choice 
must be made by the individual; 

a. ) it can gratify his/her basic needs for safety, 

love and respect, so that he/she will not 
feel threatened, autonomous, interested, 
spontaneous, and thus dare to choose the 
unknown; 

b. ) it can help by making the growth choice 

positively attractive and less dangerous, 
and by making regressive choice less 
attractive and more costly. 

13. In this way the psychology of Being and the 
psychology of Becoming can be reconciled, and 
the person, simply being him/her self, can yet 
move forward and grow (Maslow, 1972). 



Lindeman, a pioneering theorist, portrayed a new way 
of thinking about adult learning. Lindeman here 
identifies several of the key assumptions about adult 
learners that have been supported by later research and 
that constitute the foundation stones of modem adult 
learning theory: 

1. Adults are motivated to learn as they 
experience needs and interests that learning will 
satisfy; therefore, these are the appropriate 
starting points for organizing adult learning 
activities. 

2. Adults* orientation to learning is life centered; 
therefore, the ^prcqmate units for organizing 
adult learning are life situations, not subjects. 

3. Experience is the richest resource for adults* 
learning; therefore, the core methodology for 
adult eduction is the analysis of experience. 

4. Adults have a deep need to be self directing; 
therefore, the role of the teacher is to engage in 
a process of mutual inquiry with them rather 
than transmit his or her knowledge to them and 
then evaluate their conformity to it 

5. Individual differences among people increase 
with age; therefore, adult education m^ist make 
optimal provision for differences in style, 
time, place, and pace of learning. 

It is interesting to note that Lindeman did not 
dichotomize adult versus youth education, but rather 
adult versus "conventional** education, thus implying 
that youth might learn better, too, when their needs and 
interests, life situations, experience, self^concepts, and 
mdividual differences axe taken into account. 

Knowles presents the skeleton of a lifelong learning 
model. This model consists of several assumptions and 
elements; two are mentioned. 

The first assumption is that the purpose of education is 
the development of competencies for performing the 
various roles required in human life. The first element 
in the model is a taxonomy of these roles and their 
required competencies as shown in Table 1 . 

The second assumption is that the primary purpose of 
schooling is to help children and youth learn the skills 
of learning. The ultimate behavioral objective in school 
should be: "The individual engages efficiently in 
collaborative self-directed inquiry in self-actualizing 
directions**. Knowles believes these skills include at 
least the following: 

1. The ability to develop and be in touch with 
curiosities. 

2. The ability to formulate questions. 

3. H^ ability to identify data required to answer 
questions. 

4. The ability to locate the most relevant and 
reliable sources of required data. 

5. The ability to select and use the most efficient 
means for collecting the required data from the 
appropriate sources. 
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6. The ability to organize, analyze and evaluate 
the data so as to get valid answers to questions. 

7. The ability to generalize, apply, and 
communicate answers to the questions raised. 

Many researchers and theorists in both education and 
psychology have contributed information critical to 



understanding the adult learning process Carl Jung 
advanced a holistic conception of human consciousness. 
Erik Erikson provided the "eight ages of man." as a 
framework for understanding the stages of personality 
development. Maslow placed special emphasis on the 
role of "safety" in an educational sense. 



Table 1. Competency Development for Life Roles 



Roles 


Competencies 


Ixamer 


Reading, writing, computing, perceiving, conceptualizing, evaluating, imagining, 
inauirinff 


Being a self 

(with uniaue self-identity) 


Self-analyzing, sensing goal building, objectivizing. value-clarifying, expressing 


Friend 


Loving, empathizing, listening, collaborating, sharing, helping, giving feedback. 
suppoftinK 


Citizen 


Maintaining health, planning, managing, helping, sharing, buying, saving. loving, 
takinff respo:(Sibility 


Family Member 


Caring, participating, leading, decision-making, acting, "conscientizing". discussing, 
having Dei spective (historical and cultural) 


Wocker 


Career planning, technk^ skills, using support, giving supervision, getting along with 
other people* cooperating, planning, delesatinK. mana^ins 


Leisure Tune-User 


Knowing resources, appreciating the arts and humanities, performing, playing, relaxing, 
reflectine. plannine. riskine 



Lindeman. a pioneering theorist, portrayed a new way of 
thinking about adult learning. Knowles presents the 
skeleton of a lifelong learning model, on which to 
build. From these and others adult learning is 
investigated and understood 

Creating an Environment 

The third problem in providing educational 
opportunities for the AT-RISK adult is in creating an 
environment to meet the educational needs of this 
population. To understand the complexity of developing 
such an environment three examples are provided 
relating to type of adult learner previously defined. 

The Association for Retarded Citizens (ARC) offers 
funding for group housing . work environments, and 
education for the Developmentally Delayed adulL In 
Colorado there are Independent Living Centers. Center 
for People with Disabilities. Atlantis Community, and 
Denver Center for Independent Living as well as a 
variety of private facilities. 

The Colorado Literacy Assistance Center (COLAC) has 
developed tutoring programs and a statewide telephone 
networic. Hotline. In March. 1990. COLAC will hold a 
fonun to discuss deflnition and possible legislation to 
assist statewide literacy programs. 

Psychiatric outpatient, inpatient. long and short term 
facilities assist those with psychological problems 
nationwide. The forms of care vary as does the 
educational environment. The immediate concein. for 



the AT-RISK adult is one of connection between 
therapy and learning. 

There are limited learning opportunities available for the 
AT-RISK adult. The population and the need are vast 
This complexity constrains planning, funding and 
operation of an appropriate environment for the AT- 
Risk adulL 

Use of Technology 

The fourth problem in providing educational 
opportunities for the AT-RISK adult is utilizing 
technological tools to enhance learning. Children have 
many advantages in method of insuiiction. Computers, 
interactive video, and other technological advances have 
been developed to meet the educational needs of children 
with disabilities. Occasionally, on university campuses, 
in public libraries, or job settings technology is 
available to the adult learner. Unfortunately, there are 
few specific settings accessible to the I>eveIopmentaIIy 
Delayed, illiterate, or psychologically troubled adult 

After discussing four areas within this complex problem 
of providing educational opportunities for the AT-RISK 
adult it is clear that many overlapping variabl(.s exist. 
These problems are very real and seriously effect the 
lives of hundreds of thousands of adults around the 
globe. It is critical that solutions be explored, created 
anddevek)pcd. 

Solutions 
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Determining solutions for the lack of provision of 
educational opportunities for the AT-RISK adult is a 
complex endeavor. Four possible solutions will be 
described: 1) review and compare methodologies utilized 
in countries around the worid, 2) consider instructional 
design as an alternative research paradigm, 3) examine 
(he possibilities of Outdoor and Computer Education for 
Adult Needs, and 4) examine the possibilities of using 
an affective domain instructional model within the 
Outdoor and Computer Center. 



Review and Comparison of Wortdwida 
Models 

Alexander Charters and his associates (1981) have 
compiled research comparing adult education worldwide. 
To better understand the complexity of providing 
educational opportunity for an diverse AT-RISK 
population three examples of adult education are 
described. Adult education in Australia, Burma and 



Table B. Characteristics of Comparative International Adult Education 





AUSTRALIA 


BUkmA 


IRAN 


DEFINrnON 


Functional Literacy, 
Recurrent Continuing 
Education 


Literacy Training, 
Nonformal 


Literacy Training, 
Nonformal 


PURPOSE 


Liberal Aits, Social A 
Community change, skill 
improvement in labor 


literacy is flrst, functional 
q)plication is secondary 


actional community-based 
literacy 


SUBJECTS 


Post secondary education 


No aee restrictions 


overase 11 


OPPORTUNITY 


community and university 
based, private and public 


vast, group pressure is 
used to take pec^le to 
literacy classes 


vast, 6,000 communities, 
government organized 


RESOURCES 


non-government agencies, 
community-based 
movements as learning 
exchanges 


Central Literacy 
Committee, Political Pany 
& Business leaders are 
decision makers 


Minister of Education: 
large national center, large 
field operations 


LIMITS 


no federal planning, 
direction, or coordination, 
only statewide (much 
variety) 


Education gets les^^ money 
than agriculture, industry 
or transportation 


regional diversity in 
population, 

programming instruction 



Iran are briefly described in Table 2, according to six 
categories. These three countries ..ere chosen by Chris 
Duke in his study, Australia in sAsia-Comparison as 
Learning, Each country was compared on several 
variables, including these six. All of the above 
information is described in detail in Duke*s paper and 
offers many conclusions. It is interesting to note the 
variety of the subjects in each country and the 
perspective that government and business take in each 
country. This research documents a wealth of data, not 
the least of which points to the large diversity in 
population need and programming challenges for adult 
learners. 



instructional Design/Alternative 
Research Paradigm 

Systematic instructional design contains a cycle of 
detailed phases. There are many models and many 
methods to accomplish each phase. The basic five step 
model will provide a scaffolding to develop instruction 
for the AT-RISK adult. This model includes Analysis, 
Design. Development. Implement^uon and Evaluation. 



Malcolm Knowles describes a group of techniques 
developed for Westinghouse Managers Training in The 
Adult Learner: A Neglected Species. The starting point, 
front end analysis, asking Westinghouse management to 
develop a list of essential competencies required to 
perfonn*the role of General Manager. A list of 123 
items were grouped into six traditional areas: marketing, 
engineering, manufacturing, finance, personnel and 
general management. Then six task forces were 
f^tablishcd, one for each of the functional areas. These 
task forces, chaired by an assortment of management 
personnel, determined what and how much a general 
manager needs to know in the specific function areas. 

O.C.E.A.N. Ranch 

The Outdoor and Computer Education for Adult Needs 
(O.C.E.A.N.) Ranch is being developed in Colorado to 
meet the educational needs of all adults; normal and 
those with special needs. The environment, a 
combination of outdoor and computer learning centers is 
an experimental design. Research has clearly 
documented the educational potentials of outdoor 
education (Lappin. 1984; Robb, 1983; Moblcy, et.al., 
1988). Equal educational success has been found using 
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microcomputers as an instructional tool (Heerman, 
1986; Davis & Marlow. 1986; Thomas. 1983). The 
O.C.E.A.N. Ranch is in the development process. 
Within the first phase, "analysis" the following steps 
arc underway to insure need: 

1 . Needs assessment: special needs. AT-RISK and 
normal adult learners will be interviewed and 
surveyed. 200 throughout Colorado. 

2. Curriculum development: review curriculum 
models within the four areas (basic and life 
skills outdoor skills, and the arts) and SO 
instructors in each area will be interviewed, 
throughout Colorado. 

3. Assessment development: review of 
assessment techniques within the four 
curriculum areas and interviews with SO 
evaluation experts, throughout Cblorado. 

4. Analysis and interpretation of all information 
collected 

Affective Domain Instructional 
Development Model 

The affective domain is often refered to as the attitude a 
learner may require prior to learning intellectual skills 
and verbal information. Martin and Briggs (1986) 
describe Interaction as a process analyzed by "audit 
traits". These are single guides for sequencing 
experiences leading to attitudinal change. Within the 
"auJit traits" the attitude to be acquired is related to 
other skills that facilitate acquisition. Martin and Briggs 
also have reported that cognitive behavicM' has affective 
components. Gagn6 (I98S) defines affective domain as a 
choice of a course of personal action toward an object, 
person or event 

Within the scope of the affective domain an 
instructional development model will be designed and 
studied within the O.C£.A.N. Ranch environment 

Conclusion 

It has been established that there is a serious lack of 
provision of educational q)portunities for AT*RISK 
adults. Many underlying problems relate to this lack of 
opportunity; four have been discussed. Determining 
solutions for the lack of provision of educational 
opportunities for the AT-RISK adult is a complex task. 
Many possible solutions exist. It is essential to pursue 
both an understanding of the problems and solutions 
evolving around the AT-RISK adult learner. 

References 

Anderson. S.. Ball. S.. Murphy. R., & Associates. 

(1973). Encyclopedia of Educational Evaluation. 

Washington: Jossey-Bass Publish^. 
Ayres. Leonard. (1911). Open-Air Schools, New York: 

Doubleday. Page & Co. 
Beland. R. (1984). The Therapeutic Recreation 

Professional: Microcomputer Applications. Journal 

of Physical Education, Recreation and Dance, 

April. 



Brinkerhoff. R. (1987). Achieving Results From 
Training. Washington: Josscy-Bass Publishers. 

Brookfield. S. (1986). Understanding and Facilitating 
Adult Learning, Washington: Jossey-Bass 
Publishers. 

Burton. Rl. & Blakely. B. (1981). Breaking the Cycle 

of Failure Through Experimental Education. The 

Forum, 7-9, Fall. 
Charters. A. & Assoc.* .^ es. (1981). Comparing Adult 

Education Worldwide, Washington: Jossey-Bass 

Publishers. 

Donaldson. G. & Goering. O. 0971). Perspectives on 

Outdoor Education: Readings, Dubuque. lA: Wm. 

C. Brown Co. Publishers. 
Education and the Out of Doors. (1963) American 

Association for Health. Physical Education and 

Recreation, Washington D.C. 
Environmental Education in the Public Schools; A 

Special Report for the Research Division of the 

National Education Association. (1971). National 

Park Service. 

Freeberg. W. & Taylor. L. (1961) Philosophy of 

Outdoor Education. Minneapolis. MN: Burgess 

Publishing Co. 
Gagn6. R.. Briggs. L.& Wager. W. (1988). Principles 

of Instructional Design. New York: Holt. Rinehart 

& Winston. 

Hammerman. W. (1980) Fifty Years of Residential 
Outdoor Educations 1930-1980, It's Impact on 
American Education, New Yoik: Doubleday. 

Harris. D. & Bell. C. (1989) Evaluating and Assessing 
for Learning. New York: Nichols Publishing 
Company. 

Knowles. M. {l91i).The Adult Learner: A Neglected 

Species, London: Gulf Publishing Company. 
Kowalski. T. (1983). The Organization and Planning of 

Adult Education. Albany. NY: Slate University of 

New York Press. 
Kozol. J. (1970). Death at an Early Age. New York: 

Bantam Books. 
Lappin. E. (1984). Outdoor Education for Behavior 

Disordered Students. Washington: National 

Institution of Education. Department of Education. 
Mager. R. (1973). Measuring Instructional Intent. 

Pitman . Belmont, CA: Learning. Inc. 
Manion. M. (1985)., CAI Modes of Delivery and 

Interaction: New Perspectives for Expanding 

Applications. Educational Technology, 25(9), 18- 

21. 

Marchionini. G. (1985). Computers in the Curriculum. 

Electronic Learning. 4{4), 33-36. 
Martin, B.. & Briggs. L. (1986). The Affective and 

Cognitive Domains: Integration for Instruction and 

Research. Englewood Cliffs. NJ: Educational 

Technology Publications. 
Jan.Pratt, D. (1980). Curriculum Design and 

Development. New York: Harcourt Brace 

Jovanovich. Inc. 
Robinson. F. (1982). The Bradford Papers, proceedings 

from the 1982 Institution of Innovations in 

Camping and Outdo(^ Education with Persons Who 



139 

1 cr ? 



Rudd 



arc DisabkA Vol. XII, 1983, ERIC Document 
ED231565. 

Robb. G & Hamilton, E. (1985)* Issues in Oiallenge 
Education and Adventure Programing: Papers from 
the Bracfford Woods Think Tank. Indiana 
University. 

Thomas, S* (1981). Experimental Learning and the 
Handtc^CTed: Reports from the Field. The Forum, 
3^. 



Trachtenburg, R. (1985). Computer Applications for 
Decf Education. BOCES Program for the Hearing 
Impaired. North Merrick, NY. ERIC Document 
ED276243* 

Victor, K. (1981) Hiking With Troubled Kids. The 
Forum, 10-12, Fall. 



ERLC 



CHOOSING OPTIONS FOR INSTRUCTIONAL DESIGN TOOLS 



Sherwood Wang, University of Colorado; Dana Wunderlich & P. Dudiastcl, McAir Corporation 



Introduction 

Instructional design is a technology that has matured 
over the past three decades but which remains difficult 
to apply by inexperienced personnel without formal 
training in the field. Yet a great deal of course 
development for the training requirements of the 
military, of public organizations, and of industry is 
actually carried out by such personnel, with the result 
that the materials produced are often of very unequal 
quality. This problem leads to time and cost 
consequences which can greatly impair the training 
capability of these organizations. 

Different aj^roaches to alleviate this problem have been 
proposed in the instructional R&D community. The 
principal one is to develop expert systems which can 
assist in the instructional design process. Other 
approaches include template aids, design tool boxes, and 
authoring macros within computer-based training 
authoring systems. 

In the process of examining ways to improve the course 
development cycle within an on-going development 
context (specifically, CBT course production for aircraft 
maintenance training), we have been involved in an 
examination of the practical value of these approaches 
as useful instructional design tools. We consider in this 
paper the options we have examined and describe the 
approach we finally adopted. The latter led to the 
prototyping of a storyboarding tool currently in use 
within the course production program mentioned above 
(and described in an accompanying paper - Wang & 
Wunderlich, 1990). 

Starting with the design of this storyboarding tool for 
the phase of course design which immediately precedes 
the ;'>rq)aration of training materials, such as inputting 
materials into a CBT (computer-based training) 
authoring system, we intend to branch out from there to 
design an up-front instructional design tool to assist in 
lesson design decision-nuiking. Our eventual emphasis 
is on knowledge-based tools which can help structure 
the instructional design process and guide the relatively 
inexperienced course developer in preparing training 
materials. 

We are building on the lessons learned from the 
practical examination of existing design aids in the field 
and tailoring our prototyping efforts to the real needs 
expressed by a course design team engaged in a large- 
scale training program. 



The Need for Instructional Design 
Assistance 

The production of CBT materials for large-scale projects 
aimed at increasing manpower preparedness within the 
public sector and industry is a lengthy and costly 
process which strains both training schedules and 
training budgets. Any improvements in the mechanics 
of the develc^ment process which might reduce time or 
cost of development or improve the quality of training 
would be very welcome. 

Instructional systems design was developed into a 
strong technology in the 1960s and 1970s to structure 
the process of course development. This technology, 
albeit still in the process of evolution and refinement 
(Reigeluth, 1983, 1987), has been applied with success 
to large-scale training development. The ISD 
technology, however, is one which itself requires 
training and experience to apply correctly to novel 
training requirements which have not been previously 
met or which are emerging as the operational 
environment increases in technological sophistication 
and in complexity. 

Many large-scale training development efforts are 
manned only partly by course developers with advanced 
training and experience in instructional design. A fair 
proportion of course development is actually 
accomplished by personnel who might have had prior 
experience in the training field (for instance as an 
instructor) but who have not had to handle sophisticated 
training problems involving individualized interactive 
instruction as made possible by computer-based training 
and the newer technologies. Because of this, there is 
great variability in the quality of initial materials 
produced for training, which leads to the associated need 
to correct and otherwise improve these initial materials 
so as to meet quality criteria for effective training. The 
time and costs associated with this situation can mount 
quickly and thus put a serious strain on the 
organization's training capability. 

The core problem thus being addressed in the pro, ct 
described here is the lack of quality consistency in the 
initial training materials being prepared in large-scale 
training programs for the public sector and industry, as 
well as the high costs associated with the preparation of 
these materials. 

Options for Instructional Design 
Assistance 

The goal of any instructional design tool is twofold: 
Orst, to assist the instructional designer, especially the 
inexperienced one, through the provision of a more 
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slniciured design context which can guide the actual 
preparation of instructional materials; second, to provide 
ease of use, which will enhance the smooth flow of the 
designer's train of thought as he or she engages in 
instructional design. There is a balance to be achieved 
here between structuring the process enough to attain 
uniform quality on the part of many designers of 
unequal competence, and nevertheless having a user- 
centered system which enables the taking of initiative 
and a responsible creative approach to design. 

State of-the-art technology in computer science, and 
particularly in the area of artificial intelligence, can be 
profitably applied in this area. AI is already being 
applied successfully to the development of intelligent 
tutoring systems (Wenger, 1987; Psotka et al., 1988; 
Poison & Richardson, 1988). The time is now thought 
to be ripe to likewise apply AI to the instructional 
design process, essentially in the form of expert 
systems which can advise on the proper design of 
instruction. 

Expert systems are exemplified by the IDExpert being 
developed by D. Merrill and his team (Merrill, 1987, 
1989). Such a tool requests from the instructional 
designer information regarding the specific nature of the 
training task and infers from this information guidelines 
which help define the structure of the instruction as well 
as suggestions for the planning of appropriate 
instructional interactions. We have closely examined 
this tool in its current prototype version, and believe 
that the orientation of this line of research is sound. 
However, the efforts to date are still in an early state of 
development, with the result that ease of use has not yet 
been achieved. 

Other approaches to assisting instructional design are 
template aids, design tool boxes, and authoring macros 
within CBT systems. Template aids provide skeletal 
structural elements which can be filled in or adapted to 
the specific circumstances of the instruction being 
developed. They are illustrated by the software tools 
developed to help with such tasks as task analysis, 
writing learning objectives, or storyboard authoring. 
This seems to be an easily appropriated and powerful 
approach to assisting instructional design (eg. Sullivan 
& Heiben, 1988). It is the approach we have pursued in 
more detail in our own course production process, as 
described later. 

Design toolboxes provide manipulable structural 
elements which can be combined into a CBT lesson 
which is immediately executable. They are perhaps best 
exemplified by CBT authoring systems such as Course 
of Action or Course Builder, both of which capitalize 
on the case of use of the direct manipulation iconic 
interface capability of the Macintosh. The approach 
offers the instructional designer the opportunity to 
easily create CBT lessons without extensive prior 
planning or knowledge of programming. These systems 
enable the course author to structure a lesson and fill in 
that structure with domain content in a very accessible 



way, creating on the spot a lesson which is immediately 
usable by students. The ease of use of these systems is 
both their strength and their weakness. Indeed, they 
enable quick prototyping and easy revision, thus 
facilitating trial and error in design; however, they offer 
little in the way of instructional design guidance, 
although the eventual inclusion of templates in these 
tools (Misselt, 1989) will certainly help remedy this 
situation. 

Authoring macros are structured lesson input 
instructions within CBT authoring systems which aim 
to overcome the lengthy programming involved in 
developing CBT with sophisticated presentation 
systems such as WICAT or AIS II. They are not 
instructional design aids per se, but rather programming 
aids; nevmheless, any authoring macro incorporates an 
inherent instructional design template which is 
instantiated during code production and which therefore 
structures the design process, thus by default guiding 
the instrucrional designer during lesson preparation. 
This has a high structural function, forcing tlie design 
into the predetermined template, which rather limits the 
designer's ability to adapt the design in a refined way to 
the particulars of the material. The main advantage of 
authoring macros lies in speeding up the production of 
CBT lessons while ensuring uniform quality.. They are 
quite akin to templates in this respect. 

The Practical Approach Chosen 

Our task was, and still is, to evaluate the practical value 
of instructional design aids in an intensive course 
development context, as well as to undertake the 
I»x)totype development of computer-based design aids in 
response to expressed needs of instructional teams 
involved in course development. 

Our assessment of a state-of-the-art instructional design 
expert system provided to us by a government R&D 
agency revealed that this technology, while applicable 
in theory, is just not ready yet, in the practical sense, to 
be implemented in a real-world instructional design 
setting. The main problems which this assessment 
brought to light were the restricted knowledge base 
applied to the task, the rigidity of the prescriptive 
orientation provided, and the lack of user support in 
using the tool. These are all problems which further 
development can, and most likely will, overcome. For 
now, however, the promise of a useful instructional 
design expat system remains unfulfilled. 

We realize that we have not exhausted the possibilities 
in this area and we see the need to explore the use of 
related design tools which were developed under different 
circumstances, such as the IDE developed at Xerox 
PARC (Russell et al., 1988) and the Curriculum 
Design System developed at the Alberta Research 
Council in Canada (Wipond & Jones, 1988). We plan 
on examining these and other similar tools in view of 
the practical design tasks of our on-going course 
development programs. 
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We have also set aside, for now, the available 
instructional design tool kits, essentially because they 
are very system dependent and lead to the development 
of instruction whi(:h lacks some of the instructional 
management features of the more sophisticated CBT 
systems. Until these limitations are removed, and until 
these tools offer a somewhat more disciplined approach 
to instructional design, they will not provide a useful 
set of tools for the production of large-scale training 
systems, even though they may prove to be valuable as 
CBT quick prototyping tools. 

Authoring macros present a different perspective 
altogether. They are currently in use in the training 
design program with which we have been associated. 
They aie seen^iowever, as a mixed blessing: on the one 
hand, they constrain the design process to what has been 
pre-defmed as good instructional strategy for the 
particular content to be taught; on the other hand, they 
tend to perhaps make the training materials too uniform 
and possibly unduly restrict the instructional designer's 
scope for truly refming the materials. 

What we ended up focusing on his stage of the 
project was an instructional design tool that helps the 
designer freely construct lesson designs, while at the 
same time make use of templates that can expedite parts 
of the process. The tool can thus be categorized as a 
template aid, although the larger part of its 
functionality, as we shall see, focuses on enhancing the 
ease with which the designer can put together 
instructional specifications for the production of lesson 
materials. Our eventual aim is to increase the tool's 
design functionality through the ^plication of design 
knowledge, while nevertheless preserving the primacy 
of the designer's control over the design process. Our 
goal is to keep mid-way between the extremes of a 
prescriptive expert system on the one hand and the 
looseness of a design tool kit on the other. Our 
approach has been to prototype a tool that is fully 
responsive to the percei\'ed needs of instructional 
designers while at the same time offering the 
opportunity of suggesting design possibilities as these 
are embedded in interaction templates. 

In the long term, the project is aimed at installing 
computer-based tools in a variety of training 
development projects so that program teams developing 
instruction are provided with the most productive 
development environments possible. For the moment, 
the project is oriented to the instructional needs of one 
particuiar course production program currently underway 
in the company (the MDl 1 aircraft maintenance training 
program), but the resulting tools will be generalizable 
and should be easily adapted to the needs of other 
training programs. 

Specific Approach 

Our initial approach has been to concentrate on the 
design of a storyboarding tool which can immediately 



assist the course development process in the MDll 
program. The program team currently considers 
storyboarding as the area where the greatest need exists 
for a design aid tool. 

Storyboards are an important product in the ISD 
production process for a variety of CBT situations. 
Within the lesson development phase, a storyboard 
document specifies the structure and details of the 
instructional interactions with the student (CBT frames) 
according to the training objectives and related content 
material defined for that lesson. The quality, 
consistency, and ease of development of storyboard 
documents has a significant effect on the effectiveness 
and development cost of the resulting training system. 

Several course production programs within our 
organization are using a word processing method for the 
creation and revision of storyboard documents. This 
approach has brought many practical problems to light 
which impact on quality and productivity. 

Communication is one key area where improvement is 
needed Storyboard documents are reviewed and used by 
a variety of people within the MDll maintenance 
training project Quality of the end product demands that 
storyboard documents communicate an accurate sense of 
what an executing CBT lesson will look like so that the 
multiple reviewers, as well as the lesson programmers, 
can aU prc^rly visualL'x the intended finished product. 

In particular, the training lessons being designed are a 
dynamic mix of text, audio, and graphics; however, the 
previous word processing approach only supported 
textual specifications of the on-screen text and audio, 
plus the instructions for graphic dynamics. Actual 
graphic images were referenced as an attachment to the 
storyboard document The various reviewers and users of 
this document then had to imagine how the lay-out of 
the screen and the various dynamics would look like to 
the student 

Consistency of presentation is also an important 
contributor to the quality of the instructional materials. 
Consistency helps the reviewers, and eventually the 
students, to better predict and understand what is 
happening during lesson presentation. An unstructured 
won! processing tool provides no guidance or framework 
within which to consistently present the same types of 
lesson elements in the same way. A storyboard tool 
does. 

The structured storyboard development tool which we 
have built addresses many of the problems just 
mentioned. In particular, it enhances communication via 
a mixture of text and graphics, where graphic 
representations of the display screen for each lesson 
frame are embedded within the storyboard document 
along with associated descriptive textual and audio 
instructions. 
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The initial review and editing of frames are facilitated by 
extensive navigation features which allow efficient 
hypertext-like access to any structural part of the 
document based on content (terms, graphics, etc) or 
structural type (frame types, screens, instructions, 
audio). In addition, screen representations can be quickly 
stepped through to provide a very visual sense of the 
flow of a given lesson. Both consistency and 
instructional expertise are enhanced by the use of a 
flexible template system capable of supporting both 
lesson templates embodying instructional strategy and 
frame templates which give structure and guidance at a 
more micro-interaction level. 



how the support offered by these features can be made 
available to the instructional designer on an as-needed 
basis. The essence of the orientation which is now 
developing in this respect is the concept of an 
instructional design woricstation which can provide a 
range of functional features, rather than concentrate on a 
tightly pi^scribed functionality, as is usually the case 
with individual tools. The orientation is evolving 
towards a rjuch more user-centered approach to tool 
provision, an approach which is adaptive to different 
styles and levels of competence in designers, and which 
is therefore much more empowering for the individual 
designer. 



These functional features are embodied in the 
sioryboarding tool TORUS, developed in HyperCard for 
use on Macintosh computers. The specifics of the tool 
are described in Wang and Wunderlich (1990). 

Conclusions 

Our experience in this project has confirmed to us the 
value of instructional design tools for somewhat 
inexperienced personnel involved in the production of 
training materials. We have not been able to determine 
whether our particular application has resulted in 
production time savings derived from a theoretically 
more efficient instructional design process. Too many 
other factors involved in the practical world of large- 
scale course production hampered our efforts to measure 
efficiency objectively. In our particular case, it appears 
that the instructional design process took as long as 
usual (perhaps even longer in certain circumstances) 
with the storyboarding tool. 

Two qualitative elements surfaced clearly, however, in 
our assessment of this tool. The first is the improved 
quality of the storyboards, as assessed by those who 
have to review them. The second is the general 
enthusiasm of the instructional designers working on 
the project for this ID tool. There were some reticences, 
to be sure, but overall, the tool and what it offered were 
very well received by the user group for which it was 
destined. 

The central question explored in this paper was the 
relative value of different options for instructional 
design tools. The primary issue which arises in this 
respect seems to be the trade-off involved in providing, 
on the one hand, strong enough structure and guidance 
so that the less experienced instructional designer 
produces instruction of uniformly high quality, and, on 
the other hand, enough elbow room for the more 
experienced designer to not feel constricted in what 
properly remains an applied soft technology requiring 
professional judgement and flair. 

The two basic functional requirements for a 
sophisticated ID tool, structure and opportunity, are 
thus in conflict. What needs to be considered, therefore, 
is perhaps not so much how these two functional 
features can be combined within one tool, but instead 
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The scale of the ID tool design effort has increased with 
this new orientation, but at the same time, the hope of 
a feasible solution to the question of practical value in 
real production contexts has also emerged. Exciting 
developments can be expected in this area of 
instructional design in the next few years. 
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Abstract 

The first two years of a long term project to improve 
mathematics instruction through the integration of 
computer use in 5lh through 12lh grades of five New 
Jersey school systems is described. It is based on 
structured relationships with the district's personnel at 
all levels. It is intended to affect instructional practices, 
develop instructional materials ^plicable nation wide, 
and to create successful models for university-secondary 
school relationships. 

Project Background and Overview 

Since July, 1988 Stevens Institute of Technology has 
been engaged in a project with five New Jersey school 
systems of diverse socioeconomic and racial 
backgrounds to: 

1. improve mathematics insu'uction by having 
teachers effectively use computers in 5lh - 12ih 
grade mathematics courses, 

2. develop dissemination materials for classroom 
use and in-service teacher training regionally 
and nationally, and 

3 . test a model for effective college-school system 
relations. 

A project premise is that school system-wide involve- 
ment and support is vital for successful development of 
new teaching methods, especially those involving 
technology; the project involves teachers, resource 
people, department heads, curriculum coordinators, 
principals, superintendents, and school boards. 

Project History and Accomplishments to 
Date: July 1988 • July 1990 

The Project was started in July 1988 as the primary 
activity of the Center for Improved Engineering and 
Science Education (CIESE), which was established by 
Stevens President Harold J. Ravcche in July 1988 with 
support from Stevens internal funds. 

During the fall of 1988 collaborative agreements were 
established with five socially, ethnically, and 
economically diverse school systems through 
negotiations with their superintendents, high school 
principals, and mathematics department heads. Initially 
the school systems were asked to provide only: 

• stipends for teacher involvement in summer 
workshops, and 

• released time for teacher participation in 
meetings and discussions relating to the 
project. 



Teacher participation was voluntary but it was done in 
consultation with principals and mathematics 
dq)artment heads. 

There are 30 teachers in the high school component, 
which we classify as being 9th though 12th grades, and 
22 teachers in the 7lh and 8ih grade component We are 
planning to extend the program to about 30 more 
teachers in the Sth and 6lh grades. Most teachers had 
only a moderate exposure to computers, some as 
programming teachers, and none had used computers to 
teach mathematics. Project faculty and staff met with 
the teachers at each high school as a group four times in 
the spring of 1989 in preparation for the summer 
workshops. Based on these discussions we were able to 
identify and purchase, in time for the summer 
workshop, some 50 programs potentially useful for the 
identified curricular topics and modes of use. Since then 
we have acquired over 200 programs. 

High School Component Activities 

The first 9lh - 12lh grade teacher summer workshop 
took place for ten days in July 1989. There were 4-5 
hours each day during which the teachers were able to 
check out any software they wanted to investigate and 
spend time learning it Attendance at the one or two 
presentations each day was voluntary. 

At first many of the teachers were uncomfortable with 
the loose structure of the woricshops. Some wanted to 
be told what to do and how to do it. We refrained, 
indicating that effective use of computers was highly 
personal and had to be integrated with each teacher's 
style and emphasis. We encouraged them to explore sev* 
eral programs in the topics they taught indicating that 
the programs were more or less beneficial when used in 
conjunction with different teaching styles. By following 
this advice, some teachen indicated that they explicitly 
realized some of the asp^xts of their teaching style and 
by the end of the first 3*4 days all appreciated the 
freedom and went about software evaluation 
comfortably. By the end of the summer workshop all 
the teachers were comfortable with computer based 
curriculum materials, had identified opportunities for its 
use, and were preparing to implement some computer 
based activities in the coming academic year (1989-90). 

Initially, most of these implementations took the form 
of demonstration as part of regular classroom lectures 
and group problem solving. In four of the high schools 
several classrooms were eventually equipped with a 
computer, display projector, and screen on which the 
teacher could display images to the entire class. In two 
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of these, the computer and projector were mounted on 
movable carts to be used in various classrooms. 

Two of the teachers in one school prepared detailed 
lessons for display of trigonometric functions, using 
Discovery Learning in Trigonometry developed by John 
C. Kelly and distributed by CONDUIT. This module 
took more than half the fall marking period. The 
teachers reported being able to show plots of 20 or more 
functions in each 40 minute class, resulting in having 
students see and work with many more functions than 
could have ever been possible without the computer 
generated displays. 

Another teacher prq)ared scripts showing steps in the 
solution of algebraic equations using the Mathematics 
Exploration Toolkit developed by WYCAT and 
distributed by IBM. She had the program display the 
equations and pause while the class woriced on the prob- 
lem as a group. The class entered suggestions as to the 
next step and the computer pursued that solution line. 
Students were able to see quickly the results of their 
suggestions which might not be apparent for several 
stq)s. The computer kept a trace of all the inputs and 
intermediate results, allowing for review and 
comparison of solution attempts long after the 
information on a blackboard might have been erased. 
The teacher reported greatly increased interest among the 
students with some improvement in test results. 

In a geometry class the teacher reported using the 
Geometry Proof Tutor, developed in the laboratory of 
John Anderson at Carnegie Mellon University as an 
intelligent tutoring system. It uses network repre- 
sentations of geometric proofs. The teacher reported that 
having the students be able to see that representation, as 
well as the traditional one using two column tables of 
proof steps and constructions, enhanced learning. She 
indicated that drawing, and redrawing, tliese 
representations on the blackboard as the class made 
suggestions would have taken far too long for the short 
class periods in the school. The computer was able to 
display these network diagrams, generating and 
modifying them as fast as the students could make the 
suggestk)ns and enter the data. 

This same teacher reported using one of the Geometry 
Supposer series in a one computer classroom 
demonstrating certain constructions and having groups 
of about 1/3 her class do assignment with the computer 
while the other 2/3 used conventional paper and pencil 
exercises. In a lesson thai normally took two or three 
days she was able to train the class on the use of the 
machine, have them learn t!ie material, and do all the 
exercises in the same time period that the class took to 
just learn the materials and do the exercises without the 
computer; this as the result of having the computer 
reduce the often laborious effort to develop geometric 
construction'^ with conventional drafting tools and 
supporting the measurement of angles and line 
segments. 



During the fall of 1989, President Raveche and Dr. 
Friedman visited with four of the collaborating school 
boards at one of their regular meetings reporting on the 
project and discussing Stevens* commitment to the 
project and the importance of its goals in the context of 
U.S. education. Representatives of the schools and 
many of the teachers attended these sessions. 
As a direct result of these meetings, additional resources 
have been allocated for computer integration into 
mathematics education. In Hobokcn, for instance, the 
board allocated funds for overhead projectors, liquid 
crystal computer display screens, and carts to hold the 
computer with its associated projection equipment for 
each teacher in the program. In Bridgeton, in addition to 
added school board allocations, the local business 
community has been engaged in fund raising and 
contribution efforts as a result of the publicity 
associated with President Raveche's visit. 

Seventh and Eighth Grade Component 

The 7th and 8th grade program began in the fall of 1989 
with visits by Dr. Friedman with superintendents and 
principals to identify the teachers involved. The focus of 
the 7th and 8th grade program is on word problems, 
introductory algebra, introductory geometry, arithmetic, 
graphing, and estimation. 

In the spring, visits with teachers elicited topics for 
which the teachers were interested in having the use of 
computers; software was identified and purchased in 
preparation of the 7th and 8th grade teachers workshop 
in July. 

All 32 7th and 8th grade teachers attended a two week 
workshop in July 1990 to review software and plan for 
academic year 1990-91 exploratory use of com.Dutcrs in 
their classes. This group consisted of both computer 
laboratory instructors and classroom teachers. Even 
though all taught mathematics, many of the teachers 
were not mathematics specialists, and their mathematics 
knowledge and skill were decidedly weaker, for their 
level, than that of the high school teachers. It wd:> 
evident that, while reviewing software, these teachcri 
availed themselves of the opportunity to learn and 
reinforce their mathematics knowledge and skill, often 
engaging the CIESE staff in lengthy mathematics 
review sessions. 



Funding 

In July 1989 partial funding for the 9th - 12th grade 
activities was received from the New Jersey Department 
of Higher Education Office of College-School 
Collaboration from Mathematics-Science Education 
Funds (TiUe II). The Bank Street College of Education 
joined the project at that time as an evaluator of overall 
project effectiveness, project organization, and college- 
school relations. 

In the fall 1989 IBM awarded the project computer 
equipment and extensive software for two networked 
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classrooms as part of a program to upgrade teacher 
preparedness in instructional computing. Each 
networked classroom consists of a PS/2 Model 80 file 
server and network controller, 15 PS/2 Model 30/286 
workstations, and much of the mathematics and science 
software that forms the IBM Education Scries. 

Additional support was received in October 1989 from 
the U.S. Department of Education Program Fund for 
Innovation in Education (FIE) in partial support of the 
9th - 12th grade program and expansion of project 
activities into the 7th and 8th grades. 



Teachers have to change lesson plans and 
teaching styles. 

Teachers are unfamiliar with software. 
Teachers have little time to develop, 
implement, evaluate, and revise changes in 
lesson plans. 

• There are few successful, existing models of 
the use of computers in teaching mathematics. 

At many middle schools and below, the problems are 
somewhat greater since the teachers may not only be 
uncomfortable with computers, but may not be well 
trained in mathematics. 



Needs, Opportunities, and Barriers 

Project activities are based upon the following 
premises: 

1 . the need to upgrade educational achievement of 
U.S. mathematics instruction (in an 
international assessment of mathematics and 
science educauon published in 1989 by 
Educational Testing Service (Martinez and 
Mead, 1988), American 13-year-olds performed 
most poorly among peers from five other 
countries), 

2. the effectiveness of current instruction can be 
improved through the use of computer technol- 
ogy by tcachen (U.S. Congress, Office of 
Technology Assessment, 1983 and 1988 and 
National Council of Teachers of Mathematics, 
1989), 

3. the low level of current computer use in 
mathematics instruction in U.S. schools (since 
initiating the Stevens Project in 1988, we 
found active computer integration in 
mathematics classes only in private schools in 
New Yoik City and an occasional school in 
New Jersey), and 

4. although many teachers are receptive to 
change, there are multiple barriers that thwart 
change in most school districts, and that these 
barriers may be overcome by a combination of 
internal commitment and expert outside 
intervention. 

Potenu'al improvement is not automatic; it matters how 
the technology is introduced and used. Becker (Becker, 
1989) identified some characteristics for successful 
implementation, which include use of software that 1) 
provides unique capabilities (software, hardware, 
graphics) that are not otherwise available to teachers, 2) 
are used in context of an instructional plan that enables 
students to gain insight and understanding rather than 
simyly obtain correct answers, and 3) engages the 
tcacl'er in the intellectual analysis of the computer-based 
learning activities. Some of the reasons Iter little 
integration include: 

• Few math departments can afford to buy 

computers for each teacher to use in their 

room. 



Project P'an: Six Yecr Component Cycle 
at Three Leveis 

The Project is organized by grade levels into three 
components: separate on-going programs for 9th - 12th 
and 7th • 8th grade teachers, and the one proposed for 
Sth - 6th grade teachers for which funds are currently 
being sought. 

Each of these components is planned in six year cycles, 

including five summer workshops. 

During the initial academic year, exploratory 
discussions are held between Stevens faculty 
and staff and the school system teachers and 
administrators at each of the five schools. 
These discussions determine topics for which 
the teachers would like to try to use 
computers. Stevens personnel then explore the 
software available for those topics and select 
and acquire those that appeared to address the 
needs expressed 

During the first summer, in a 10*day workshop 
at Stevens, teachers from all the school 
systems have an opportunity to become 
familiar with hardware and software, and 
compare ideas and approaches. In 1989 each of 
the 29 teachers who attended received a $1000 
stipend and expenses. 

In the second year, exploratory implementation 
of computer use and materials is initiated in 
consultation with Stevens faculty and staff. 
In the subsequent second summer workshop, 
teachers engage in extensive curriculum 
planning, preparing for more pervasive 
implementation during tiie third program year. 
These plans are documented in the form of 
lesson/topic plans authored by individuals or 
groups and distributed to all participants in the 
program. 

In the third year, teachers will try to use 
computers throughout the year whenever they 
consider it appropriate. Much of this use will 
be based on lesson plans written by other 
teachers in the program. Each teacher using 
such a lesson plan will be asked to suggest 
revisions, which will be communicated to the 
author of the lesson. After each time revisions 
are made, the lesson plan will be distributed 
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again to all the teachers in the program. It is 
expected that by the end of the third year over 
100 lesson plan for the use of computers in 
teaching mathematics will have been written, 
classroom tested at least once, and revised. 
During the following summer, in the third 
woilcshop, a team of 10-12 teachers, at least 
one from each school, will develop publishable 
lesson plans, classroom materials, and guides 
for teacher use both at their schools and others. 
Materials testing and refmement continues for 
two more years, followed by the sixth year of 
active dissemination activities. 

Evaluation 

Formal project evaluation is being done by personnel 
from Educational Testing Service and The Bank Street 
College of Education. ETS is primarily concerned with 
educational growth of the students, impact of 
technology on the teachers, effect on the social structure 
of the schools, and the overall responsiveness of the 
projects activities to the needs of teachers. Bank Street 
will concentrate on the general organization and 
administiation of the project In addition, a student from 
the Princeton Sociology Department is studying the 
project for his doctoral dissertation. 

It is premature to evaluate educational effects; so far we 
have collected pre- ueaunent test scores and measures of 
teacher attitude. 

Much of the effect on students due to a project such as 
ours may be due to changes in teachers behavior. 
Teachers pedagogical styles as well as changes to the 
curriculum are being tracked, looking particularly for 
problems teachers must overcome, coping mechanisms, 
innovations, and teachers attitudes toward computers and 
innovation which affects and is affected by the project 

Introduction of computers in mathematics instruction 
may change the social structure of the school. Data on 
bureaucratic, administrative, and social actions by the 
various participants, both direct and indirect, is being 
collected 

Assessment of the roles and activities of the CIESE 
staff as intervening agents is being performed. Question 
of advantages and disadvantages of having university 
faculty acting as change agents and how to establish 
long term communication between college faculty and 
secondary .vhool teachers are being considered. 

Some results of the summer workshops can be 
communicated. Teachers reported that time was the 
most important resource provided to them. They used 
this time to evaluate software and plan lessons as they 
were expected, but they also reported that they had time 
to meet and discuss applications, logistics, and 
strategies with other teachers. They reported that this 
happens rarely, even within their own schools. Non- 
overlapping release periods and department meetings 



devoted to administrative issues preclude unhurried 
consultation with their own colleagues. They almost 
never get to meet teachers from other sjtiools. At the 
workshops they could group around a <;ompuier to 
watch another teacher explore a program, which would 
inevitably lead to questions and then discussions. We 
provided breakfast and lunch on the same floor as the 
machines and the classrooms, so these discussions were 
allowed to continue even as the routine of the day 
progressed. 

Teachers expressed their appreciation for being treated 
like professionals. They viewed everything, from being 
provided a .stipend and food, as well as having their 
questions and suggestions being seriously considered by 
college faculty in their fields, to the workshop paradigm 
that they (the teachers) were the decision makers 
concerning software and lesson planning, as indices of 
being treated professionally by professionals. They 
complained about being constantly flooded with 
•'experts" telling them what to do; they respected 
Cl£SE*s openness to hear their needs and suggestions. 

Project Rational and Merit 

We believe the key factors for the success of the 

program so far include: 

Stevens faculty and staff role as collaborators 
and consultants. This non-directive approach 
has contributed to greater teacher self esteem 
and enhanced self-identiflcation. 
On-going suppoit services to school system 
administrators. They more readily invest in 
technology which they feel might otherwise 
receive limited use and/or have limited impact 
Association of the priorities of a major science 
and engineering college with the importance of 
precollege mathematics and science education. 

Given the rapid pace of hardware and software 
development and the distributed management of U.S. 
education among its 16,000 independent school 
districts, it is difflcult to see how appropriate expertise 
can be brought to bear on the introduction of 
technology into the schools. 

One promising model is the one we are following: to 
have faculty and staff at colleges and universities work 
closely with teachers and their entire school systems. 

There are about three thousand colleges and universities 
in the United States and many have signiflcant capacity 
for such action. This means there are about 5 or 6 
school systems for each college and university, which is 
how many schools are in our project. What 
opportunities and barriers exist for colleges and 
universities to start such projects is the subject of 
another study. 
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Appendix: Demographics of Collaborating School Systems 

The five school systems involved in this project were selected to provide a variety of educational environments in 
which to create model programs. Although the schools vary in their current level of computer usage in mathematics 
course, each school does have a sufficient number of computers available to have begun the initial implementation 
phase of this project. Some possibly relevant statistics are give in the table. 



School System 


District 
Pop. 


School 
Pop. 


White 


Black 


Hispanic 


Asian 


SAT 
Ved) 


SAT 
Nfath 


BridKcton 




1329 


41% 


48% 


10% 




397 


445 


Hobo)^ 


42.000 


1236 


18% 


9% 


67% 




308 


346 


WestSidc 


320.000 


1438 


7% 


93% 


3% 


2% 


312 


354 


Coluinbia 


38.800 


1664 


73% 


20% 


4% 


3% 


436 


497 


Tcnafly 


15.000 


842 


76% 


1% 


1% 


23% 


comb. 


1026 



In Bridgeton 55 students arc classified as ''impaired**. Less than 20% of the students enroll in four year colleges. 

Most middle class students from Hoboken attend private schools. The resulting imbalance in the public school 
system has created a pressing interest on the part of school system administrators to recapture this middle class 
population. 

West Side High School is one of 13 high schools in Newark, whose school district is the largest and one of the 
oldest school system in New Jersey and has severe social and economic problems. 

Columbia High School is in the School District of South Orange and Maplewood; 85% of the graduating class of 
1988 continued their education in degree-granting schools for higher learning. 

Tenafly high school is in the Tcnafly-Alpine school district; 84% of the Class of 1988 enrolled in 4 year colleges; 
10.6% participated in the National Merit Scholarship Program; 11% enrolled in community colleges. 

Tcnafly is a suburban community, 4.4 square miles in area and located six miles northwest of New York City. It 
consists mainly of one-family homes. The combined Tcnafly-Alpine population is about 15,000. Parents arc highly 
educated, with 46% of the fathers and 24% of the mothers holding degrees beyond the B.A, Seventy-two percent of 
the fathers and 28% of the mothers arc involved in professional and/or managerial occupations. 
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IN MULTI-SITE LOCATIONS 
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Abstract 

Knowledge is of two kinds; we know a subject 
ourselves, or we know where we can find 
information upon it. 

Samuel Johnson (1709-1784) 

Dr. Johnson would approve of the idea of a '^knowledge 
network** or public access computing. Public access 
computing involves using a computer to access 
electronically stored information about a myriad of 
topics from a host of resources. An advantage of 
electronically stored information is that the data can be 
searched and retrieved from sites distant from the 
physical location of the stored information thereby 
allowing access to information at all hours of the day or 
night John Sculley (1987), CEO of Apple Computer, 
dreams of a **Knowledge Navigator** designed to travel 
throughout the world's data banks retrieving information 
on demand in a brief q)ai) of time. Meeting Educational 
Needs TlmMigh On-line Research (MENTOR), is a step 
in realizing both Samuel Johnson's concept of 
knowledge acquisition and John Sculley's vision of the 
future. 

What Is tha MENTOR Project? 

Meeting Educational Needs Through On-Une Research 
(MENTOR): A Computer-Based System for Knowledge 
Acquisition is a project funded by the U.S. Oq^artment 
of Education that provides the technology and training 
public and private school students need to access and use 
electronic information to prepare them to function in 
the high technology environments of college or the 
work place. MENTOR is equipping and interconnecting 
two public and one private high school, an elementary 
school, a regional public library and a large research 
university into an on-line research facility that will 
provide almost instantaneous access to information 
about millions of items in their combined collections. 
The MFNTOR Project takes advantage of the graphical 
user interface cap^ilities of the Macintosh computer.^ 
MENTOR will make use of Compact Disc-Read Only 
Memory (CD-ROM) databases to provide access to 
dictionaries, encyclopedias, census data, comprehensive 
histories, scientific data, graphic and sound data from 
any school or library location. Through local school 
networks, students and teachers will work with 
computers in their classrooms to access their media 
center's on-line card catalog and CD-ROM databases. 
Through phone lines, students and teachers will be able 
to dial-up any of the other sites in the MENTOR 
Project to acquire additional information from their card 
catalogs or CD-ROM databases. 



Rationale for the MENTOR Project 

We live in an **information age** where instant access to 
data often means the difference between success and 
failure and sometimes between life and death. While 
business, industry and governing agencies have ad^tcd 
quite well to accessing and using critical information 
through interconnected networks of databases, our 
schools, for all practical purposes, are still in the **stone 
age** of data acquisition. Classrooms and school libraries 
function much as they did fifty years ago. Students 
acquire infomiation by searching volumes of print-based 
materials in the time honored traditions of scholars from 
the past The problem is that our information bases 
have grown so numerous and massive that manual 
searches can be tedious and frusu*ating, and often 
inefficient and non-productive. Elementary and 
secondary school students need to be given access to and 
training in how to use electronic technology in the 
acquisition of knowledge if they are to succeed in our 
society of tomorrow. 

Introduction 

Futurists have predicted for a long tim^^. that society is 
nu>ving toward an **Infonnation Age.** In many areas, 
such as business and commerce, the information age is 
an everyday reality. Businesses depend on instant 
information from far flung databases to make critical 
decisions which mean the difference between profit and 
loss. Our schools for all practical purposes have not 
entered the information age. Students still gather 
information using traditional search strategies involving 
the card catalog and discrete print indexes, a proces.<; that 
in no way emulates the current information retrieval 
practices used in business, industry and academic 
libraries. To graduate students from our secondary 
schools without an awareness of computer-based 
information resources and without the basic skills 
necessary to initiate a search, would be to short-change 
them. 

How will schools fare with the new information 
technologies? Students in elementary and secondary 
schools constitute the largest population of library and 
information users with library media centers being the 
main access points for information in the schools. In 
the realm of technology, few innovations have captured 
educators as completely as the microcomputer, yet 
microcomputers are only slowly beginning to be used 
for information retrieval in school settings. 
Microcomputers allow for the dramatic changes that are 
taking place in the way information is distributed, 
accessed and mastered. As ihe information hub of the 
school, the library media center is the logical promoter 
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of new technologies thai can extend the amount of 
information available lo students. The information 
contained on traditional catalog cards can be transferred 
to an on-line database creating what information 
specialists call an on-line public access catalog 
(OPAC). This on-line catalog can be accessed from 
classrooms within the school via direct link 
networking, or can be accessed from other locations 
through conventional telephone lines. 

Compact Disc-Read Only Memory (CD-ROM) 
technology, which was introduced in early 1985, has 
important implications for schools. The advent of CD- 
ROM databases not only means vast amounts of 
information can be stored on a single disc, but offers a 
new way of searching through massive amounts of 
material. More than 250,000 pages of information or 
the equivalent of 1,500 floppy disks can be stored on a 
single compact disc. With a CD-ROM drive attached, a 
microcomputer is able to search, browse, and retrieve 
literally a modest Hjrary of books, journals and indexes. 
This technology has unveiled a new arena for 
potentially innovative applications, especially in 
education. In the past, few schools have subscribed to 
remote on-line services such as DIALOG because of 
budgeting problems. CD-ROM technology offers an 
alternative to expensive remote on-line searches. 
Information formerly available only by subscribing to 
remote on-line services can now be stored on a CD- 
ROM. Information is still considered on-line, but is not 
retrieved from a remote long distance ser/ice. Since 
many CD-ROM databases are updated quarterly, CD- 
ROM is a practical, cost-effective approach to {roviding 
current information. Students can use the CD-ROM 
without worrying about search time as a cost factor. 
The compact disc is an effective medium for high 
resolution graphic images, speeches, and animation, as 
well as for textual information. By combining the 
attributes of CD-ROM technology with the networking 
capabilities now available, electronically stored 
information will be available at all times in both the 
classroom and at home after school hours. 

Purposes of the MENTOR Project 
Objectives of the Project: 

1 ) To inuoduce and instruct students in the use of 
new technologies and new ways of obtaining 
information. As a result of using the 
MENTOR System, students will learn the 
logical process required for accessing 
information electronically, thereby developing 
the research skills they will need in college or 
in the job market, 

2) To provide educators and the general 
community with an information retrieval 
environment by providing a network model 
that emphasizes the idea of lifelong learning. A 
networic that will effectively link the resources 
of the public school media centers, the public 
library, the university library and a private 
school's libraries, thereby making information 
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available both during and after hours, is a 
logical extension of a rich informational 
environment. This cooperative effort will 
allow students and the entire community to 
have access to the holdings of all institutions 
involved 

3) To provide educators with evaluations of 
hardware and software (a prototype system) for 
implementing an on-line retrieval information 
system. By utilizing the resources ol the 
Department of Instructional Technology and 
the expertise of its faculty, evaluation of the 
project will be an integral component 
throughout the implementation phases. The 
evaluation will focus on user friendliness, 
effectiveness, and cost-effectiveness. 

4) To disseminate the fmdings through summer 
seminars for educators, presentations at 
national conventions, newsletter mailings and 
articles in educational journals. 

Alvin Tofflcr (1970), author of Future Shock and The 
Third Wave, in commenting on the effects new 
technologies have on education stated, **If education has 
any ftmction, any justification, it is to prepare young 
people for the future. If it prepares them for the wrong 
future, it cripples them." (Toffler, 1981). He would 
argue that the slogan" back to basics" should be revised 
to "forward to basics." Literacy is not conHneU to print 
in a world drenched in electronic communication. 
Preparing young people to function effectively in the 
emerging economy means foreshadowing what life will 
be like in the woric force when these children are grown. 

Developing the Full Potential of Students^ 

MENTOR includes the placement of microcomputers in 
subject departments in the high schools involved (see 
Figure 1.). The students and teachers can access the 
holdings of the library media center or obtain 
information or full-text from CD-ROM databases as the 
need arises. Some refer to this as the "teachable 
moment." Skills needed in college and the job market 
can no longer be taught without the use of computeri;ccd 
information retrieval. 

Provide Sound Tested Models for Improving 
Outcomes of Educational Systems* The 
literature concerning implementation of information 
retrieval environments in school settings is sparse. 
MENTOR is important because the evaluation of the 
system will provide information that can help other 
counties and states make decisions concerning 
information retrieval systems. The transferal of 
bibliographic information from catalog cards to 
computer databases is already taking place in many 
schools. To assure that this information is transferred in 
a form that can be networked with other schools and 
libraries and accessed from home microcomputers is of 
prime concern. Because of the cooperative nature of the 
project, assurances that the holdings of all libraries can 
be accessed is of utmost consideration. The pooled 
n^sources of public and private schools, as well as those 



Multi-Site Knowledge Networks 



of the public and university libraries* will provide a is to contribute to existing knowledge by addressing the 
comprehensive network of available resources. problems inherent in implementing any system. 
Educators need knowledge and guidance in the purchase 
and implementation of programs. A goal of this project 




Figure 1. Athens Academy MENTOR Site conHguration. 



Extent of NMd for the Project 
Needs Addressed by the Project: Power On! New 
Tools for Teaching and Learning (1988)» a report from 
the Congressional Office of Technology Assessment 
(OTA)» states thai if technology is to realize its 
potential* attention should be focused on four closely 
related areas and that each area is affected by the ethers. 
The areas are: 

• expanding the amount and capability of 
technology in schools to increase student 



• pn>Wdingtndtiing and siq>pcrt for teachers 

• supporting research* development* 
demonstration* and evaluation* with emphasis 
on ties between research and the classroom 

• encouraging innovation and improvement in 
educational software 

The first three needs are q)ecific£lly addressed by 
MENTOR. The Office of Technology Assessment 
report also states diat limited research results suggests 
that students using databases outperform odier students 
in tests of informati<Hi processing skills and in addition* 
teachers repon that students using databases understand 
underlying concepts and relationships better* woriced 
more cooperatively* and become more enthusiastic 
about gathering and analyzing data (Power on!^ 1988). 

How Needs Were Identified. The need for such a 
project as MENTOR can be established by witnessing 
the changes that have taken place in other areas of 
society. Business* the military* higher education* 
medkine* all have entered the age of information with 



great magnitude. A great chasm exists between whai is 
possible and what is actually taking place in school 
settings. In the Public Broadcasting series* **The Day 
the Universe Changed: A Personal View by James 
Burke* " the major events in history that have ^hasped 
our civilization are presented. In one episode Burke 
describes the significance of Gutenberg's invention of 
Ae printing press and the inqnct it had on society as a 
mechanism to disseminate infonnation. Buike believes 
that the computer as a technological force will have a 
much greater influence. He states* "We're about to 
bump up against a revolution that will make what 
Gutenberg did kxric like a quiet affernoon stroll** (Burke* 
1985). The literature in general* substantiates the need 
for schools to provide expmences that will prepare 
students to live in an information society. Particularly 
noteworthy are the works of Daniel Bell (The Coming 
of the Post-Industrial Society), Alvin Tofiler (Future 
S^iock and the Third Wave), and John Naisbitt 
(Megatrends). Yet surveys* incbding one srat to media 
^)ecialists in the state of Georgia, indicate that while 
many media ^)ecialists and educators would like to use 
microcomputers for a broad range of implications* 
including information retrieval* most feel inadequate 
about their knowledge and training. The need for 
training is clearly established. The University of 
Georgia has an on-line computerized catalog as do most 
large academic libraries. Commonly w^A indexes such 
as ERIC, Psyclit, Dissertation Abstracts, and Newsbank 
are now available in CD*ROM format and are being 
used in llie University of Georgia library system. To 
exempt elementary and secondary schools from 
providing experiences that prepare students to use 
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libraries other than their local school media center 
would be an abdication ol responsibility. 

How Needs arc Being Mel by the Project. MENTOR 

addresses the folIo^^^ng needs: 

expanding the amoui*:; and capability of 
technology in schools to increase student 
access - MENTOR provides students with a 
computer-based system for searching the holdings 
of their individual libraries as well as the indexes 
and full-text information available on CD-ROMs. 
Students and teachers are able to access these same 
resources from selected classrooms. Students using 
the Athens Public Library in the evenings, will be 
able to access the information in any of the school 
sites. If one has access to a microcomputer at 
home, the databases of the libraries involved in the 
network can be accessed via a modem any hour of 
the day or night The holdings of the University 
of Georgia Library are also accessible in this 
manner.. This information-rich, easily accessible 
environment is expanding not only student access 
to information, but is doing the same for the entire 
community. 

providing training and support for 
teachers - MENTOR also addresses the need for 
providing training and support for educators. Two 
faculty members a:;d a graduate assistant from the 
Department of Instructional Technology are 
work'ng with media specialists and teachers during 
the installation phases, as well as during the 
training phases of the project. ] Workshops for 
teachers and media specialists are planned so that 
MENTOR can be successfully integrated into the 
curriculum. Since the Department of Instructional 
Technology provides courses and workshops for 
media specialists throughout the state, another 
avenue fcH' training exists. 

• supporting research, development, 
demonstration, and evaluation, with 
emphasis on ties between research and 
the classroom - Research on the end-user of 
information retrieval systems has usually 
concentrated on college students. Research is needed 
on how elementary and secondary students interact 
and benefit from computer-based information 
retrieval. Tracking of student users through 
observations and interviews, in addition to 
quantitative measures of use, will be an important 
task during the evaluation phase. The locations 
involved in the project will inevitably be 
demonstration sites for those who are planning to 
implement their own systems. Visit and talk to 
installed sites" is a suggestion reiterated in the 
literature on information retrieval. Formative 
evaluation of the system, its reliability and its 
effectiveness, is taking place on a continual basis. 
A formal summativc evaluation is planned at the 
end of the second year. 

• encouraging cooperation and the sharing 
of information resources - The cooperative 
efforts of the institutions involved is providing the 
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community with a richer information base from 
which to choose. With the increasing numbers 
electronic CD-ROM databases, no one library can 
hope to p^'rchase all the on-line databases that their 
patrons mii^ht request. By coordinating acquisition 
of these databases, MENTOR will facilitate the 
sharing of information resources, thereby 
encouraging cost-effectiveness. 

Benefits Gained by Meeting the Needs. Studies 
have indicated that ease of use is the single most 
important determining factor for the use of information. 
In a recent article in Book Report, the author (Pincock, 
1989) describes what an instant hit a CD-ROM 
reference disc was with high school students"It 
represented a major leap from having to comb through 
seven volumes of Readers' Guide to a few minutes 
using the computer to search across five years of 
magazines. A similar search using printed sources 
would demand several hours and few students would 
have such tenacity. Most would have been satisfied with 
what they found in a single volume, without ever 
uncovering the article of a year or two ago that might 
have significant information for their project. For 
public schools to ignore the impact and societal changes 
that current technology has afforded would be unwise. 

MENTOR is meeting the needs of an information 
hungry community. Through the use of multiple CD- 
ROM players and databases such as Grolier's Electronic 
Encyclopedia, Xiphias', Time Table of History: Science 
(Sc Innovation, and Merriam-Websier's Ninth New 
Collegiate Dictionary, students are beginning to 
experience thrill of instant access of information. This 
is just the "tp of the iceberg" and the convenience of 
accessing these databases and others from the classroom 
or home is attractive. 

There is another important benefit embedded in 
MENTOR. The lines of communication are being 
opened among the MENTOR Project Team members. 
Connecting the concerns of a private school, the public 
schools, the public library system and the College of 
Education's Department of Instructional Technology is 
indicative of community action in its best form. 

Plan of Operation 

Project Design. MENTOR is being conducted using 
the resources of two school systems, a regional library 
and The University of Georgia. The location of the 
study is Clarke County, situated in Northeast Georgia 
approximately 70 miles east of Attanla (See Figure 2.). 
Clarke County with a population of 88,000, is the 
smallest county in physical size in Georgia. Athens, the 
county seat, is the home of The University of Gcorf'ia 
and many environmentally sound industries. The county 
has a consolidated school system with two high 
schools, three middle schools and ten elementary 
schools. In addition, there are three private schools, 
Athens Academy, Athei;s Christian and St. Joseph's 
Elementary. This area of Northeast Georgia is 
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experiencing steady population growth and three 
additional elementary schools are presently under 
construction in Clarice County. In addition to the 
materials contained in the elementary and secondary 
school libraries, the Athens Regional Librcry and The 
University of Georgia Libraries provide excellent 
resources. The thrust of MENTOR is to provide easier 



and quicker access to resource information contained in 
these dispersed repositories. Through easier access, 
students aie being encouraged to research projects more 
carefully and in greater detail, thus mastering the critical 
skills they will need to flourish in an information 
society. 
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MENTOR is a "Knowledge Netwofk** placed in three 
hi^ idiools, an deoeniiiy sdKX>L poblic li^ 
leteaidittmversiQr. Using MENIXxtsmdents at any 
site can access a weakh of infonnatkm indoding 
dectnmc caid catak>gs and Contact Disc Read Onty 
Memofy(CD^(^ Databases. Teachers and studems 
wiU use ME^^OR to improve their knowledfe 
acqmsiiion skills and wpply this knowledge to solving 
problems. 



Figure 2. MENTOR "Knowledge Networic Site Locattons. 



MENTOR is linking the libraries of Clarke Central 
High School* Cedar Shoals High School, Athens 
Academy Upper and Lower Schools* the Athens 
Regional LibiSffy and The University of Georgia. The 
decision was made to emphasize infonnation retrieval at 
the high school level, althou^, Athens Academy 
Lower Scbod htt; been inchided to test the feasibility 
infonnation access in the tower grades. The lower 
scbod also was selected because Athens Academy is 
situated in a campus-like environment with a numbor of 
classroom buildings dispersed over a five acre site. 
This beautiful, campus-like environment, while 
conducive to scholarship* sometimes restricts access to 
information from the upper or lowor school libraries 
because of their sq)arate tocatknis. 

Administration of the Project. MENTOR is 
scheduled to run fcH* a two-year period The management 
plan calls for a multi-track implementation process. 
During the first year, the CD-ROM databases have been 
selected and hardware installed at each site. Initial 
testing of the on-line CD-ROM database system is 
taking place in Clarke Central High School and 
University of Georgia sites prior to installation in the 
other locations. All sites have installed computer 
hardware and are woricing toward placing their library 
holdings into a computerized database. The conversion 
of the card catalog to an electronic database is being 
done commercially. The library media specialists in 



each school are wcHking during the first year to make 
their "electronic card catalogs" fully op^ational. The 
implementation of a fiilly int^rated liteary automatton 
system which includes an on-line catalog, as well as 
aq)abilities for accomplishing the tasks of acquisition, 
circulation* inventcvy and catatoging, will take a year to 
accomplish. After the University site became 
operational, teachers and library media personnel bom 
all schools involved in the project attended a one week 
training workshop to familiarise themselves with the 
system. During the woikshop training, teachers were 
not only shown how to operate the hardware, but 
worked in teams with the project director and evaluation 
specialist to create student activities for integrating 
MENTOR into the curriculum. Teachers involved in 
this project included science, social studies and language 
arts feculty as well as \ibmy media specialists from the 
three high schools. 

The second year will be spent evaluating the 
effectiveness of the on-line systems and making 
modifications necessary to facilitate the operations. 
Teachers will be encouraged to add additional activities 
that utilize MENTOR. All activities identified and used 
by teachers will be compiled into a handbook and 
operations guide to be used by future teachers and shared 
with other schools interested in adopting an on-line 
"knowledge network.** This operations guide will be 
used as an integral part of workshops scheduled during 
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the second year of operation. During the second year, 
two one-week workshops will be provided for media 
personnel from across the state of Georgia to familiarize 
them with MENTOR and to help them implement such 
systems in their home school districts. 

At the end of the second year of MENTOR Project, all 
hardware and software will be turned over to the 
respective school systems with an agreement that 
MENTOR continue in operation and be expanded into 
additional schools as funds become available. Ongoing 
and yearly evaluations will track the success of 
MENTOR and hopefully such systems will become an 
integral part of the schools. 

Evaluation Plan 

Evaluation is an important aspect of the MENTOR 
Project. Both formative and summative evaluation 
techniques are being used to validate MENTOR'S 
success. Althou^ all hardware and software items used 
for MENTOR are "off the seir items, this is still a 
jHX)totype system. Formative evaluation of the system 
has already resulted in the modiflcation of automation 
and network software. MENTOR'S operation and 
continuing evaluation will no doubt result in further 
changes to the **neiworiL** Both equipment and software 
logs are being maintained at each site for noting 
operational idiosyncrasies. In addition, a technical 
specialist is on call to assist with any operational 
poblems. 

Patron usage is also being closely monitored. Items to 
be evaluated include; levels of use, type of use, user 
friendliness of the system, teacher and student attitudes 
and teacher evaluation of the CD-ROM databases. 
Teachers arc working closely with the library media 
personnel in evaluating the selected databases and in 
making recommendations for additional titles that can 
be placed on-line. 

Appropriateness of Methods. The ultimate goal 
of a retrieval system is to bring about a desired change 
in the knowledge state of the user resulting from the 
content of the retrieved information. Therefore while the 
evaluation plan is concerned with the efflciency and 
reliability of MENTOR itself, concentration is being 
placed on the effectiveness of the system from the user's 
point of view. The following are a partial list of the 
criteria for evaluating the system performance: 

1) utilization rate 

2) users' studies 

3) user satisfaction 



4) retrieval effectiveness as measured by user 
feedback 

5) effectiveness of end-user searching 

Although qualitative performance criteria are not easily 
measurable, there are a number quantitative criteria that 
reflective the quality of service. Frequency data are a 
quantitative measure of system use that is indicative of 
the acceptance of the system by users. 

Summary 

MENTOR is a functioning prototype of the "knowledge 
network" that must be placed into our schools if we 
hope to keep pace with the growing mass of 
information in our society. MENTOR is teaching both 
students and teachers new ways to master information 
and use it to enhance research, learning and pleasure. We 
can no longer expect students to retain in "memory" all 
of the facts they need to know. Education must provide 
a student with the ability to find information and 
MENTOR is designed to meet this critical need. 
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THE EMERGENCE OF THE EDUCATIONAL TECHNOLOGIST: 
A STANDARD FOR CERTIFICATION 



Jamie J. Kline, Lehigh University 



Abstract 

This study will provide an analysis of the role of the 
educational technologist by examination of 
contemporary practice. From this analysis, a proposed 
set of standards for certification of educational 
technologists will be developed. 

The /tudy will provide answers to the research questions 
thro»-.gh the use of survey instruments directed to 
personnel in technologically advanced public school 
districts. Information will be sought requesting the 
identity of technologically advanced public school 
districts which currendy have persons responsible for 
the promotion of educational technologies in the district 
curricula. States which have certiflcated persons 
perfonning the duties of educational technologists, or 
their equivalent, will be asked to forward their state 
certification requirements for the position. States which 
do not certify persons as educational technologists will 
be asked to present information regarding positions of 
an similar nature (such as instructional technologist). 

The survey instrument will supply information which 
will be used to produce a set cf guidelines for state 
agencies and teacher preparation institutions to certify 
educators as educational technologists. 

Introduction 

The introduction of new electronic technologies has 
changed the nature of instructional design and delivery 
(OTA, 1988). "The new technology is symbolized by 
computers, cable television, and satellite data 
communications" (Johnson, 1981, p. IS). 
Technological applications in education in recent years 
have been largely influenced by the proliferation of 
microcomputers in the schools as a new medium for 
delivering information (Mackay, 1988; Cimochowski 
1984; Belland, 1982). Microcomputers, coupled with a 
variety of sophisticated audiovisual media, present 
challenges to those responsible for design and 
implementation of instruction. So called "convergent 
technologies" have created tJie need for a new breed of 
specialists with expertise in a variety of educational 
technok)gy applications areas (Allen, Dodge, and Saba, 
1989, p. 47). Accordingly, an educational technologist 
might be described as an educator who uses modem 
scientific methods and electronic instructional delivery 
systems to enhance teaching and learning. The name 
"educational technologist" implies most of the 
following special skills: curriculum construction; 
educational administration; design, production, and 
evaluation of instructional materials and/or programs; 
media utilization for education (i.e. television. 



computers, etc); library and information science; and 
select areas of psychology (i.e. cognitive psychology) 
and sociology (i.e. diffusion and adoption of 
innovations) (Hutchenson & Kankin 1989, p. 6; 
Berbeckar, 1986). Additionally, Wa^^^oner and Goldberg 
(1986) recommend the technologist possess skills in 
instructional systems design. 

Typically, the responsibilities for promoting the use of 
educational technologies in schools have been assigned 
to indiviJuals filling dual roles such as mathematics and 
science teachers or library/media specialists who possess 
litde or no formal background in educational technology 
but nonetheless are required to function in this capacity 
(Bratton and Silber, 1984; Kerr 1989). Teachers, need 
specialized skills in using modem electronic educational 
systems to function in the classroom both now and in 
the future (Kerr 1989). 

According to a ^tudy conducted by Emest (1982), 
educational media competence in teacher preparation 
programs has been made necessary by three basic forces: 

1. the educational profession has been subjected 
to critical scrutiny by members of its own 
ranks as well as the public sector as emphasis 
on accountability and cost effectiveness has 
increased. 

2. a rapid proliferation of technology has created 
an ever it)creasing array of tools and techniques 
of communication, entertainment, and 
instruction. Teachers must cope with students 
who are "media literates," 

3. there have been myriad research studies 
conducted on the effectiveness of educational 
technology in the teaching/learning proccss(p. 
2) 

Content area teachers cannot reasonably be expected to 
keep up with the rapidly changing trends of modem 
information and educational technologies, as well as 
their own teaching areas (Belland, 1982). Assuming 
there is general agreement with most of the research 
described in the literature review, a distinct need emerges 
to assess the potential impact of educational technology 
with respect to the personnel responsible for its 
promotion i" schools, and propose minimum standards 
for their certification. 

Stahl (1986) described teacher certification as "the 
validation by a state board of education that a person has 
the required educational course work, experience, and 
professional characteristics deemed necessary to instruct 
in a given area" (p. 16). In order to certify educational 
technologists as professional educators, state and local 
education agencies must address the need for criteria 
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consisienl with established certificari- a and staffing 
procedures (Galey and Grady, 1987). 

The purpose of this study is to determine the duties and 
responsibilities most important for those personnel 
assigned to promote the use of educational technologies 
in technologically advanced public school districts, 
regardless of their position title, by incorporating the 
various recommendations found in the literature review. 

The Problem 

The literature indicates that educators responsible for the 
promotion of educational technologies in public schools 
have no set of standards on which to base their 
occupational duties and responsibilities. Furthermore, 
there are no states with regulations governing the 
school districts in this regard. School district personnel 
performing the duties of educational technologists, must 
be legitimized under the guidelines of a specialist's 
certificate or similar professional designation. The 
literature also points to a lack of uniformity with regard 
to the title assignments for practitioners in this area of 
expertise. 

Need for the Study 

The continued growth and development of educational 
technologies as well as their expanded use in the 
classroom, have created some potential problems for 
educators. Several areas of concern were identified in 
1986 in documents published by the Association of 
Educational Communications and Technology (AECT) 
which outlined leadership imperatives for the 
educational technology profession. The AECT 
document described a distinct lack of proper descriptive 
terminology for practitioners in the field: 

...there are too many names used to refer to our 
profession: educational media, instructional 
media, instructional technology, educational 
communications, instructional 
communications, learning resources, etc. These 
names are definitely confusing to people 
outside our profession. We must provide 
leadership by recommending a single 
terminology to designate who we are (AECT, 
1986, p. 14) 

The association further recommended that the profession 
develop "...clear standards for certification of its 
members." (p. 15) 

This study will provide an analysis of the role of the 
educational technologist by examination of 
contemporary practice. From this analysis, a proposed 
se^ of standards for certification of educational 
technologist'^ will be developed. It will be necessary to 
identify the duties and responsibilities of persons 
promoting the use of technologies in technologically 
advanced school districts, and compare and contrast their 
occupational activities with respect to the various titles 
assigned them. 
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Research Questions 

The study will be guided by the following research 
questions: 

1 . What titles are used to describe the positions of 
persons who are responsible for promoting the 
use of educational technologies in selected 
technologically advanced public schools? 

2.. What are the occupational duties and 
responsibilities of the persons described above? 
Which of the duties and responsibilities are 
most common among them? 

3. To what extent were practitioners' 
competencies obtained through: 

a. collegeAiniversity for-credit courses 

b. inservice workshops, demonstrations 

c. on-the-job iraining/self help? 

4. What state mandated certification standards are 
currently in place for persons described above? 
What elements are common an[K)ng them? 

5. What educational technology competencies are 
considered to be the most important in meeting 
the duties and responsibilities of respondents, 
performing the function of technologists in 
selected technologically advanced public school 
districts? 

6. According to the respondents above, how 
frequently are content attnbutes, suggested in 
the literature, included in the competencies 
outlined above? 

7. What educational and training qualifications do 
the respondents above consider to be essential 
in the preparation and certification of persons 
who serve as educational technologists? 

8. How do the respondents' percq)tions of critical 
competencies for serving as educational 
technologists compare with those found in the 
titerature? 

9. What competencies are recommended to 
provide a model for professional certification of 
the educational technologist? 

Limitations of the Study 

The need to review various areas of the educational 
technologies is necessary because of a lack of in-depth 
studies with regard to professional certification of 
persons functioning as educational technologists. 
Related areas such as computer certification, 
library/media specialist certification, and teacher 
preparation in general, will provide basis for this study. 
The study will therefore be limited to those issues 
which directly affect educational technology as it applies 
to professional certification of public educators. 

The investigation will concern only school districts 
which have been identified in the literature as being 
exemplary in their efforts to integrate advanced learning 
technologies^ including microcomputers^ into school 
programs. Furthermore the methods of identifying such 
school districts rr.ay be flawed, and therefore districts 
which meet the minimum criteria identified in the 
literature may not be included in this study. 
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Because of the various names applied to the positions 
described herein, this study may have missed some 
individuals responsible for the promotion of educational 
technologies in selected technologically advanced public 
schools. 

Competency items for the questionnaire were gathered 
from several studies mentioned in the literature review, 
and since some of the competencies pertain to 
technologies which may be considered obsolete, efforts 
made to ensure that additional competencies representing 
the newer technologies may have been short sighted in 
scope and therefore missed some important areas of 
concern. 

Current Research 

This review of the literature will identify pertinent 
information with respect to the competencies necessary 
for the perfOTmance of the duties and responsibilities of 
educational technologists. Findings considered 
important to the answering the research questions of 
this study will emanate from expert professional 
opinion and the most recent studies and surveys 
conducted at the local, state, and national levels across 
the United States. 

The microcomputer will have a great deal of influence 
in future teather certification practices and as such will 
be included in any discussion of certification of 
educational technologists (Galey, 1980). Furthermore, 
the trend toward greater use of non-print media in the 
classroom will ultimately result in increased emphasis 
on teacher preparation in these areas as well (Galey and 
Grady, 1987). 

Diem (1982) pointed out that..."without adequate 
training and understanding there is little likelihood of 
most teachers augmenting instruction with technology 
no matter how attractive its usage may seem" (p. 3). 
For these reasons this study will also review teacher 
preparation program course content and inservice 
programs as they relate to technology use in the 
classroom. 

Many of the studies used the Delphi technique to obtain 
consensus for the competencies listed in the 
questionnaires. The siuvey instrument question items 
generated for this study were formulated from this 
review and sent to the appropriate parties in state 
government and technologically advanced school 
disuicts. 

Topic areas to be covered in this review include the 
following: 

1 . Teacher certification 

2« Educational technology competencies 

3. Computer technology competencies 

4 . Educational media competencies 
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Teacher Certification 

Issues of teacher certification are handled differently by 
each state, but the basic premise of certification is that 
of societal approval and respect for the accomplishments 
of individuals who meet the qualifications for the 
certificate in question. While competence is the reason 
for certification to exist, there arc two other vehicles for 
assurance of competence: accreditation and licensure. 

According to Bratton and H-ldcbrand (1980): 

"...accreditation is the process whereby an 
agency or association grants public recognition 
to a school, college, imiversity, or specialized 
study program that meets certain predetermined 
qualifications or standards." (p. 22). 

With this kind of voluntary, self regulation, an 
institution will conduct a self study and then be visited 
by a team of professionals from the same area of 
expertise to conduct a type of audit* of the institution to 
determine whether or not it meets the goals of its 
charter as stated. .Accreditation is only for institutions or 
their programs, not individuals. 

Licensure "...is a mandatory legal requirement for 
certain professions in order to protect the public from 
incompetent practitioners." (p. 22) Individuals only are 
licensed, not the institution or employer. 

Certification ..."is the process by which a professional 
organization or an independent external agency 
recognizes the competence of individual practitioners." 
(p. 22) 

The authors point out that teacher certification is more 
of a licensing process since it is issued by a legal 
agency, the state government, and requires that all 
representatives possessing this credential shall have 
completed certain requirements prior to its issuance. 
Endorsements are added as necessary when the licensee 
desires to teach in specialized areas such as a computer 
specialist, or media specialist 

The literature indicates that the emphasis placed on the 
educational technologist's time, other than teaching, can 
be classified into three main competency categories with 
respect to educational applications: (1) educational 
technology applications (includes instructional design), 
(2) computer te:hnology applications, and (3) audio- 
visual media applications.. This review will present 
studies from both areas in an effon to describe the 
minimum competencies required by personnel working 
with these technologies. 

Educational Technology Competencies 

Ernest (1982) conducted a study to determine what 
educational technology competencies arc most necessary 
for the preparation of new teachers at the preservice 
level. It also looked at a sample of teacher preparation 
programs to determine if there were curricular 
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provisions set aside to include such competencies. The 
study posed the following questions: 

1 . What educational technology competencies for 
teachers are identified as most important by 
experts in the fields of media and teaching? 

2. What educational technology competencies for 
teachers are identified as most important by 
personnel in a selected sample of teacher 
education programs? 

3. What is the relationship between the 
competencies identified by the experts and the 
competencies identified by the personnel in the 
teacher education programs? 

4. To what extent are the identified competencies 
taught in the selected sample of teacher 
education programs? 

5. What is the relationship between the 
competencies identified by the experts and the 
extent to which the competencies are taught in 
the selected sample of teacher education 
programs (p. 7)? 

This study was conducted using a modified, two round 
Delphi technique to identify competencies. The data was 
then used to provide basis for assessment. 

The Delphi panel consisted of fifty experts in the media 
and/or teaching fields. Twenty media specialists from all 
over the United States, and ten each from the areas of 
teacher education, supervisory/administration, and 
classroom teaching, all from the state of Alabama. 
Members were selected according to their publication 
records, research or other application areas in educational 
technology. The final group was selected from an initial 
group of 100 by random process. 

The study findings resulted in two basic conclusions: 
(1) media specialists, teacher educators, administrators, 
and teachers perceive educational technology 
competencies for teachers as highly important; and (2) 
graduates of the preservice preparation programs in 
Alabama have not utilized educational technology at the 
levels of importance established by the experts' and 
institutional representatives' perceptions (p, 19). 

On a scale of 1.00 to 4.00, 64 of the 69 competencies 
had a mean of 3.00 or higher on round two. This placed 
the items in '*^c moderately important to very important 
range. Interestingly, the five items below the 3.00 mark 
dealt with microcomputers, telecommunication/ 
computer developments, and photographic processes. 
Three quarters of the competencies were perceived as 
very important for preservice by Alabama teacher 
education program heads. Two of the microcomputer 
competencies listed were rated less than "moderately 
important" by the same group (p. 19) 

Ernest discovered that preservice preparation was found 
to be at a lower level of importance than perceived"No 
category was taught in more than 44.6% of the 
institutions, and no single competency was taught in 
more than 57.1% of the institutions" (p. 19). 



Kerr (1989) presented a paper to the Association for 
Educational Communications and Technology entitled: 
Technology, Teachers, and the Search for School 
Reform. This paper attempts outline the basic elements 
of reform necessary for teachers and educational 
technologists to realize maximum return on their efforts 
to include technology in the curriculum. Kerr reviewed 
the f ^Ids of teaching and educational technology from 
the points of view of the teachers and the technologists. 
Four elemental areas were presented: (1) preparation of 
models fcH' teaching-with-technology; (2) design of 
intelligent software; (3) creation of technologically 
based tools to support teachers* professional work and 
development; and (4) improvement of research about 
technology in education (p. 5). 

The educational technologist's vision of educational 
technology has seen the gradual trend away from 
emphasis on devices to a focus on systematic 
instructional design models and procedures along with a 
steady growth of interest in the cognitive and behavioral 
principles of learning, including artificial intelligence. 
Kerr points out that as a result of this approach by 
educational technologists, their activities have become 
increasingly distanced from the woik of ordinary public 
school teachers. He suggested that there is general 
agreement among educational technologists that if 
teachers would incorporate instructional design 
practices, classroom experiences would be vastly 
improved. 

...students would become motivated, 
instruction would become clear and logical, 
student achievement would increase, teachers 
would be freed from the drudgery of routine 
tasks, and classroom activities would become 
more varied(p. 7) 

Kerr says, however, that teachers have been slow to 
respond, either to those who have encouraged them to 
use technology-as-hardware or those who advocate the 
use of instructional design as a possible solution to 
pioblems in education"The expectations and hopes of 
technologists have changed; reality, in the main, has 
not" (p. 7). 

The teacher's vision of educational technology still, for 
the most part, implies hardware with its associated 
software, but not a process approach such as 
instructional design offers. Many teachers use simple 
technologies such as showing films or video tapes to 
get relief from the daily rigors of the classroom setting. 
Often, the use of films or tapes may bring on new ideas 
or alternative points of view just from seeing how the 
rest of the world operates. 

According to Kerr, if teachers did want to pursue the 
educational technologies from the standpoint of 
development, other than the application of hardware and 
software, they would have little or no time to spend 
because of the constraints placed on them by 
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administrative maintenance. FurthemKM'C, "teachers are 
not provided with or taught how to use more helpful 
alternative models during teacher training"* (p. 9). 

The technologist's view of teaching gives the 
impression that teacher's roles in education ..."is 
something to be refined and shaped by principles of 
instructional design: inconsistencies are to be smoothed 
out, digressions eliminated, predictability developed" (p. 
10). Instruction, according to the technologist, should 
be designed in a manner that minimizes the contribution 
of the teacher and, in some cases, eliminate the 
instructor altogether. 

The teacher's view of teaching changes rapidly during 
their early exposure to the task. Teachers, once they 
make it through theu* first year or two, begin to 
formulate their ideas about what it is they should be 
doing (and how) including the construction of their 
classroom "world" which ultimately reflects their 
concept of what teaching entails. Kerr points out that 
teachers find comfort and solace in their classrooms in 
helping to overcome feelings of isolation from their 
peers. They often resent outside interference in their 
classroom affairs, and consider most administrative 
requests to be an unnecessary burden on their already 
demanding schedules. Teachers' descriptions of their 
occupation tends to direct attention to conventional 
"wisdom of practice** and the need to develop and 
maintain individual interactions with their students. 
Kerr states that teachers ..."see the barriers to 
improvement of schools as lying largely in 
administrative realms and express their frustration with 
not being allowed to have more control over their own 
destinies" (p. 8). 

Educational reform, according to Kerr, presents new 
challenges to both teachers and educational 
technologists to stem the tide of decline in academic 
standards and lack of confidence in public education. He 
cites the Teachers College Record article by Joseph 
McDonald who described the current "second wave" of 
reform as the search for "the teacher's voice" (p. 9). The 
major goals are: 

1., Democratize school adminisu-ation. This will 
allow teachers to take certain leadership 
responsibilities in an effort to direct their own 
school's destiny 

2. Enhance teacher professionalism. Allow 
teachers the ability to select curricula, 
instructional materials, and teaching methods; 
make decisions on research, peer evaluation, 
merit promotions, and professional 
development 

3. Alter teacher and student roles. Shift roles to 
allow the teacher to become a guide or coach 
instead of the source of knowledge, diversify 
classroom activities to reduce frontal 
orientation of instruction, and introduce a 
variety of instructional models and practices. 

4 . Conduct new kinds of research on teaching and 
teacher preparation. Attempt to have colleges 



work more closely with teachers in defining 
what teacher preparation courses should cover 
in their curriculum. 

Educational technology can do its part in educational 
reform by develc^ing leaching- with-iechnology models, 
designing supporting software, supporting the education 
and future professional growth of leachers, and 
improving research on teaching-with-technology (p. 1 1- 
13). 

A Delphi study entitled. Preparing Schools for the Year 
2000, (1988), conducted in cooperation with the 
Association for Childhood Education International, 
surveyed the attitudes and opinions of specialists on the 
use of educational technologies in their curricula for the 
foreseeable future. The group was asked to prognosticate 
the use of technologies in the K-9 classrooms of the 
future. In addition, they were asked to forecast how 
curricula might change as well as instructional materials 
designs in the coming years. 

This Delphi study started with an open ended set of 
questions in order to elicit a wide response. Three basic 
questions were asked of the respondents: 

1. In reviewing the K-9 curriculum, what are the 
top five skills and/or subjects that are likely to 
be emphasized in the next ten years? 

2. What types of instructional materials will be 
essential in the classroom and in the school 
media center in the mid-90's? 

3. In what ways will classroom teachers use new 
technology as a teaching tool in the mid-90's? 
(p. 1) 

The seventy eight participants consisted of classroom 
teachers, school and college level library/media faculty 
or specialists, school administrators, state education 
representatives, educational consultants, and national 
education organization personnel (ERIC, NCLIS). 
Twenty-seven states from all regions of the United 
States were involved 

The results found professionals in general agreement 
that "today's students must be educated to face a world 
that is becoming increasingly complex: a highly 
technical and information-oriented society where 
individuals must absorb an explosion of information as 
well as new theories and discoveries. The rapid advance 
of computer technology has fueled this process, making 
vast amounts of information available to those who 
have learned to use a computer" (p. 3) 

Further comments pointed out the increasing dependence 
on technology to carry out even simple life skills and 
how students "must be prepared to manage and control 
it " The world was described as **fast moving, constantly 
shrinking" and "influenced by a variety of cultures that 
we don't know very well" (p. 3). Important on the list 
of skills to be developed by students arc things such as 
comprehension, communications,and cooperation. 
Furthermore, coping skills to deal with the myriad of 
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social problems in today*s society were given high 
priority by respondents along with a need to become 
literate, critical thinkers. 

The classroom of the year 2000 will look like a high- 
tech living room complete with televisions, VCR's and 
computer gear interconnected to videodisc players and 
CD-ROM drives and satellite receivers to provide 
maximum access to a wide variety of multimedia 
experiences for students. An educational consultant 
remarked that, "Technology will help bring the world 
much closer to all classrooms. The learning 
environment will not be the classroom, but the world" 
(p. 4). 

The curriculum in the year 2000 is expected to contain 
such important skill requirements as listening, 
speaking, reading, and writing in the language arts; 
problem solving in math and science; social skills and 
personal fitness in health and physical education; 
literature and art appreciation in the humanities and arts; 
and geography and history and world cultures in the 
social studies. 

Finally, teachers need a whole r v set of applications 
skills to develop new attitudes toward use of technology 
in the classroom of the year 2000. 



way of research studies. Suggestions made by the 
authors are indicative of such inquiry: 

1. Identify those institutions that have exemplary 
teacher education programs in computer 
education. Then, once identified, a listing can 
be circulated through national organizations 
with widespread distribution networks on a 
regular basis. The dissemination of this 
information at the national and state levels 
"...would be of great value to teacher training 
institutions as well as prospective teachers and 
inservicc teachers." (p. 4) 

2. The requirement for computer education course 
woric in teacher training programs may revolve 
around a state education agency (SEA) policy 
issue since most state departments of education 
set teaching certification standards. In reports 
by the Education Commission of the United 
States (1983) only a small number of states 
have set a computer literacy requirement for 
high school graduation and even fewer (the 
District of Colombia, North Dakota, and 
Virginia) are requiring teachers to have course 
work in computer education. However, another 
thirty states are studying the teacher 
certification requirements and/or recommending 
computer education training for teachers. 



Computer Technology Competencies 

Cimochowski (1984) conducted a study to examine 
levels of prescrvice and inservice teacher preparation in 
computer education at member institutions of the 
American Association of Colleges for Teacher 
Education (AACTE). The study investigated: (1) what is 
being taught in introductory courses; (2) which of the 
departments/schools are teaching computer education 
courses; (3) how many courses are being offered for 
preservice and inservice teachers; (4) qualifications of 
the professors teaching the courses; (5) number of 
students enrolled in the courses; (6) time prescrvice and 
inservice teachers spend using computers and; (7) 
relationship between computer education courses offered 
and size of the teacher education program, public or 
private school affiliation, and regional location of the 
AACTE member institutions (p. 1). 

Findings indicated that there are a number of questions 
which remain to be answered: 

1. What can be done to implement computer 
education course work at institutions where no 
such programs exist, to broaden the scope of 
those programs which do exist, and to make 
both preservice and inservice teachers aware of 
what teacher training institutions offer in 
computer education? 

2. What can be done to increase the number of 
teacher training programs that require computer 
education course work for graduation? (p. 3) 

Solutions to the problems outlined above are 
multifaccted and complex, but they are forthcoming by 
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The author further recommends that this type of 
investigation be conducted on a regular basis as a 
follow-up using similar questions with revisions as 
necessary. She feels that it would be valuable to ask 
about such things as upper level course offerings, and 
include a description of each to determine course 
coverage and content, number of credits. Additional 
information could be items such as which courses are 
required, course time spent on computer related 
activities both in and out of class, etc. 

Chen (1984) conducted a Delphi study on computer 
related competencies needed by secondary teachers in 
Taiwan. This study was conducted to identify a list of 
competencies needed by teachers through the year 2000, 
and to determine differences in the assigned importance 
of these competencies by the respondents involved. The 
Delphi approach was used because there was little data 
available related to the minimum computer related 
competencies needed be the secondary school teachers of 
Taiwan. 

^hen surveyed secondary school principals and teachers, 
teacher educators, and experts from government and 
industry from various geographical locations across 
Taiwan. The results were rank ordered as follows: 

1. Possess ability to solve simple computer 
related problems. 

2. Have a thorough understanding of the 
application of the computer to motivate 
student learning. 

3. Possess ability to effectively utilize the 
computer as an instructional aid. 
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4. Understand the influence of the computer, its 
impact on modem society, and its future 
trends. 

5. Possess ability to operate at least one 
commercial microcomputer or personal 
computer in school or at home. 

6. Possess skills in using instructional packages. 

7. Have knowledge of computer as«»sted 
instruction (CAI), its meaning and functi. 

8. Possess ability regarding the ^plication of the 
computer to input/edit/output test from test 
inventory. 

9. Have ability to motivate students* interest in 
studying computer science and to apply 
computer concepts creatively to a variety of 
applications, (p. 99) 

Conclusions reached as a result of this study were as 
follows: 

1. The competency statements show that the 
panel members are very optimistic, future 
oriented, and aware of the problems teacher 
education programs will have in incorporating 
computer technologies in their courses. 

2. The competency statements range from the 
basics, such as use of the computer, to newer 
issues of automation and artificial intelligence. 

3. It was the consensus that competencies as 
presented represented the majority of expert 
opinion. 

4. Secondary principals and teachers placed the 
most items in the agree or strongly agree 
category, indicating their desire to see these 
items become part of professional education 
courses. 

5. Government and industry experts as well as 
teacher educators perceived more competencies 
in the 'disagree* or 'strongly disagree* levels 
than principals or teachers. It is believed that 
this is a result of their concept of need with 
regard to the competencies identified in the 
study being different *han the other 
respondents. 

Among the final recommendations of this study was 
one suggesting that ...*'niany competencies are beyond 
the basic levels needed by all secondary school teachers, 
but should be possessed by at least one teacher on every 
campus** (p. 120). A further suggestion included having 
an office of computer education in each school to 
facilitate computer use by teachers. Office duties would 
include: 

1 . Supervising computer laboratories 

2. Assist in developing scope and sequence of 
computer education at the school 

3 . Set up equipment for use by faculty and staff 

4. Train faculty and staff in selection and use of 
software 

5 . Troubleshoot hardware and software problems 

6. Coordinate repair of equipment 



Other recommendations of this study pointed out some 
barriers to computer literacy training of teachers: 

1. The traditional focus of teacher education 
programs 

2. Lack of instructional computer facilities 

3. Lack of competent teaching staff 

Some respondents pointed out that...**the lack of 
knowledge and support from administrative staff were 
the largest obstacles preventing the use of computers in 
schools** (p. 121). It was suggested that computer 
literacy be extended to administrators, and teachers 
through continuing education, inservice workshops, and 
graduate credit courses. 

Audio-Visual Media Competencies 

Savenye (1989) conducted a study to determine what 
teachers need to know in order to use technology 
effectively in the classroom. The study was done in an 
attempt to improve the content of a one credit media 
competency course required of teachers in order to 
receive their credentials in the state of California. The 
course provides teachers with the opportunity to apply 
the principles of instructional systems design according 
to *'...Gagn^*s (1977) nine events of instruction; media 
equipment operation; production of dittos; overhead 
transparencies and laminated visuals; media selection; 
basic instructional video production; and an introduction 
to educational computer software evaluation" (p. 3). 

Survey questions were asked to determine both present 
and future needs of teachers for technology use in the 
curriculum. Areas of coverage were as follows: 

1. Entry level technology skills of teacher 
credential candidates: What instructional 
materials have they produced? What devices can 
they q)crate? 

2. Final evaluations of the skills they learned in 
the media course by students finishing the 
course: What do credential students perceive as 
the most, and least valuable skills they 
learned? 

3. Perceived needs of new teachers in the field: 
What technology related skills, learned during 
their credential programs, do they feel help 
them teach most effectively? 

4 . Perceptions of teacher education leaders: What 
do teacher educators believe teachers should 
learn in order to teach effectively using 
technology? (p. 3) 

The results show that teachers, both preservice and 
inservice, believe that learning about commonly 
available mat^^rials and equipment is important. Such 
items as overhead transparencies, dittos, chalkboards, 
and using projectors of all kinds were included in their 
opinions. It seems, however, that knowledge of most of 
these items was already possessed before the media 
course began, and that valuable class time could have 
been spent learning about newer, more distant 
technologies such as video production and media 
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selection. Teichcrs fell thai things already known could 
be applied .0 lesson preparation techniques in methods 
courses. F irther recommendations were to increase 
emphasis ou items such as video production, computer 
related activititr inr^L'ding CAI, interactive video, CMI, 
CD-ROM, and telecommunications. 

In summation, the author concluded that: "with the 
great need for teachers who are adept at performing the 
multiple roles required of them when they use the power 
of technology in their classrooms, can provide an 
opportunity for teachers to become 'empowered' 
themselves. This study indicates that preservice and 
inscrvicc teachers recognize the dual need to be skilled at 
basic media use, while preparing themselves to 
skillfully use the new technologies to enhance their 
students' learning" (p. 8) 

McCutchcon (1984) conducteo a .study to determine the 
factors influencing the content of introductory media 
courses offered in undergraduate teacher education 
courses in the United States. The two major areas of 
investigation were (1) the topic coverage in the courses 
and (2) the perceptions of the course instructors as to 
how such topics are determined. It was discovered that, 
for the most part, the course instructors themselves 
were largely responsible for course content with 
contributing factors such as: college or department 
policy, student expression of needs, slate certification 
requirements, items reported in the literature, and 
textbooks playing a lesser role. 

McCutcheon points out that ..."the importance of 
introductory educational media courses to the 
professional community offering them cannot be 
disputed. Such courses provide an overview of the 
instructional technology field and by so doing, serve the 
function of recruitment both into additional courses and 
possibly into careers" (p. 10). The introductory course 
serves as the largest employer of teachers in 
instructional technology. These teachers in turn add 
their expertise to further course development add the 
study of new technologies to their course outlines as 
they become viable for curriculum integration. 

The media course content items are among the items 
listed as important in a number of other studies cited in 
the literature. Some studies date back to the I930's and 
contain some competencies that are Uiily futuristic for 
the times. An example is the mention of a study by 
Stames in 1937 which describes as one of its 
competencies "radio and television education" (p. 32). 
Current studies have added more modem devices to the 
study of educational technology as computers, video 
discs, CD-ROM, interactive video, and satellite 
telecommunications systems. 

Methodology 

The study will provide answers to the research questions 
through the use of survey instruments directed to 
personnel in technologically advanced public school 
districts and teacher preparation institutions which have 



educational technology programs. The districts in 
question will be identified using criteria established by 
SoniCi* (1989) as well as the state departments of 
education. Information will be sought requesting the 
identity of technologically advanced public school 
districts which ciurcntly have educational technologists 
or their equivalent in place. Those states which have 
certified persons performing the duties of educational 
technologists, or their equivalent, will be asked to 
forward a copy of their state certification requirements 
for the position. States which do not certify persons as 
educational technologists will be asked to present 
information regarding positions of an similar nature 
(such as instructional technologist). 

Population 

Each state department of education will be asked to 
supply the names of two school districts who they 
identify as exemplary districts with regard to the 
application of advanced technologies to their curricula. 
Further identification of districts will be done according 
to the following criteria presented in the literature 
(Sonier, 1989): 

1.. The district has been cited by popular literature 
as being innovative in the integration of 
microcomputers and other technologies in 
school instruction. 

2. Evidence of planning, adequate funding, and 
staff/community participation is cited in the 
literature review. 

3. The literature citation of the school district's 
technological accomplishments appears in a 
publication specializing in educational research 
and/or technology (p. 41) 

Once the districts have been identified, it will be 
necessary to determine the name(s) of those who are 
performing the tasks of the educational technologist or 
whose occupational title most closely compares with 
that of educational technologist as outlined in the 
literatme. 

Gathering Data 

The survey instrument will be sent to the contact 
persons in those school districts identified as 
technologically advanced. Demographic information 
will be requested as part of the questionnaire to establish 
background for the study. In addition, the respondents 
will be asked to indicate which of the (number) 
competency areas is currently part of their position 
responsibilities, and the extent to which they perceive 
the value of the area in question. 

The instrument 

The instrument will be developed based on findings 
from the literature as well as information gathered from 
state departments of education which have certification 
for educational technology related occupations. 
Competency items for the questionnaire were gathered 
from several of the studies mentioned in the literature 
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review. The competencies in those studies are 
understandably dated and therefore every e'^^'ort was made 
to ensure that additional questions representing the 
newer technologies were included. If there are items 
which the respondents feel are important but have been 
omiucd, there will be blank space for them to write in 
additional compctciKy areas as needed. 

A Likert scale format will be used to facilitate timely 
completion with a pilot version being distributed 
locally to test validity. This pilot version will be 
administered to all ten school districts in Northampton 
County, Pennsylvania, to assess their perceptions of the 
educational technologist's duties and responsibilities. 
Comparisons will be made between Jie pilot and the 
actual descriptions provided by the literature and the 
certifying states to determine need for revisions. 

Analysis of Data 

The demographic information will be analyzed to 
indicate the following: 

1 . percent 6[ position titles 

2. percentof years of experience 

3 . percent of highest degree attained 

4 . percent of major and minor study fields 

5. percent of credit or non-credit (inservice) 
courses 

Each of the (number) competency areas will be analyzed 
for the following: 

1. The frequency of respondents affuming the 
presence of a particular competency area 
necessary for their present occupation. 

2. The average of the importance "value index" 
assessments placed on a particular competency 
area. 

3. The percentage of content attributes present in 
the respondent's competency area necessary for 
their occupation. 

4. Pattern of response for each competency area 
with respect to the respondent's position title. 

Survey results will be reported for each competency 
area. Results of reported competency area occurrence as 
well as the frequency of perceived competency area 
content attributes will be reported as raw scores and 
percentages. 

Questions assessing the necessity of a competence area, 
when present in the respondent's occupational 
responsibilities, will be in a Likert scale format. A 
numerical value will be assigned to each response, for 
example: a value of S will be assigned to indicate that 
the competency area is of "great importance", 3 to 
"some importance", 1 to "no importance". These results 
will be averaged and labeled as the "importance value 
index". 

Competency areas reported as important for over two- 
thirds of the respondents will be recommended for 
consideration by state certification officials in the 



generation of a certificate classification. A rank order 
listing of the competency areas most frequently 
addressed by the respondents will be delineated in table 
form. How important these areas are perceived to be 
will be reported to provide certification competencies for 
the state agencies. Sample occupation description 
documents, collected from the exemplary school 
districts, will be examined for the presence of content 
attributes identified in the literature. This analysis will 
help verify the accuracy of the respondents' perceptions 
of their district function for the position they occupy, 
and identify exemplary occtq)ational competencies. 

The second pan of the insuiiment will present criteria of 
the educational technologist's position description as 
determined by the literature review and certifying state 
criteria. The respondents will indicate the 
appropriateness or inappropriateness of certification 
attributes based on his or her perceptions of their 
present occupation or function. Responses to the 
questions will be in the form of a Likert scale. Two sets 
of questions will be developed from the initial questions 
below. The pilot set of questions will be distributed to 
the appropriate persons in the Northampton County 
school district in Pennsylvania to determine validity. 
The participants' answers will determine the content of 
the final set of questions. 

Discussion 

This study will be conducted over the next six months, 
and results will be reported shortly thereafter. It is 
important to realize that the field of educational 
technology is in a state of continuous flux and any 
investigation of this kind must be ongoing. The 
competencies determined to be most important this year 
will undoubtedly take on a different level of emphasis 
next year. Most of all, this kind of effort is important 
from the standpoint of professionalization both for the 
educator and the certifying agencies preparing them. It is 
hoped the results of this study will pave the way for the 
establishment of standards with which the profession 
can claim its rightful place in the worid of educational 
excellence. 
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USING EXPERT SYSTEMS APPROACHES 
TO SOLVE MEDIA SELECTION PROBLEM: MATRIX FORMAT 



Fing'Cheng Chao, Peter J. Legree, Philip D. GiiUs, and Michael G, Sanders, U. S, Army Research Institute 



Media selection has been identified as an integral part of 
the total instructional development process in many 
instructional design and development modeIs(Anderson, 
1983; Gagn6, 1988; Briggs. 1981; Boucher. 1973; 
Slcenian, 1979). However, the selection of efficient and 
effective media "is complex and difficult because it is 
based upon a combination of interreL'^ted factors", 
(Anderson, 1983 R 1). 

Many systems have been developed to help instructional 
designers select media. The problem with these systems 
is that they may overwhelm the instructional designers 
with the quantity of information they produce. This is 
important because many courses must be designed by 
instructional designers in as little as three to twelve 
months. Complicated manual systems may not be 
usable for instructional designers under these severe 
time constraints. The instructional developers simply do 
not have the lime to run through complicated 
flowcharts, matrices, or worksheets. 

Three additional problems with paper-based media 
selection models are that they do not reference high 
technology media, present viable and thorough media 
selection factors, and do not reflect the knowledge of 
multiple media specialists.. 

The complexity of the manual media selection models 
and the consistency of many of the media selection 
factors suggest that a MSES could be developed to 
improve the entire process. Such an expert system 
would assist instructional designers in making quality, 
and timely decisions. This research project was designed 
to address these two issues by developing and evaluating 
a Media Selection Expert System (MSES). 

The Problenfis and Its Setting 

According to media experts, media selection decisions 
should be based upon the instructional setting, learner 
characteristics, insuiictional events, learning outcomes , 
physical features and practical factors (Reiser and Gagn^, 
1983; p. 15). Unfortunately, instructional designers 
may not be media specialists and hence not aware of 
many of the factors needing to be considered during 
media selection. Thus, it is often difficult for 
instructional designers to make the appropriate media 
selection decision. Furthermore, assistance during the 
media selection process is often difficult to find. 

Most media selection models don't include high 
technology media. For example. Intelligent Tutoring 
Systems, Digital Video Interactive, and Compact Disc 
Interactive were not included in any of the fifteen media 
selection models located during a literature review of 



media selection procedures. Most models also do not 
justify the specific media and media selection factors 
(attributes) that they reference, (e. g. Briggs, 1981; 
Romiszowski, 1988; Boucher, 1973) 

The format for the display of media selection models 
can be categorized into flowcharts, matrices, and 
worksheets. Most of the media selection expert systems 
available today use the flowchart s^roach to develop a 
rule based expert system. The flowchart procedure leads 
to a progressive narrowing of media alternatives. 
Questions about media selection are posed in a 
particular order, and as each is answered, the number of 
candidate media is reduced. Typically, flowchart models 
produce a small set of recommended media. 

Although the flowchart format is usually simpler to 
use, the approach may be criticized because the 
questions and their sequence is usually based on only 
one human expert's knowledge. Thus the flowchart 
format is subject to the problems of personal bias and 
the limitations of individual knowledge. In contrast, the 
matrix approach allows the knowledge of a group of 
human experts to be easily stored and lets the user 
choose media ba5ed on the experience of more than one 
expert. The MSES's knowledge base was constructed in 
a matrix format because of these considerations. 

Purpose of the Research 

The puqx)se of this research is to develop and evaluate a 
media selection exr^ system that will address all of the 
above problems mat are embedded in current media 
selection models. The expert system should assist 
instructors and instructional designers in making 
consistent and systematic decisions based on the nature 
of the learning task. 

Importance of The Research 

Richard Clarke (1983, 1985), has come to the 
conclusion that "no media, in their own right, have any 
influence on learning effectiveness, but are mere 
'vehicles' for more, or less, well designed instruction". 
But, vehicles vary in the pleasure they provide and in 
their efficiency. Questions such as, "Is the medium 
capable of presenting the instructional stimuli required 
for learning?" and "Can it arrange for the students to 
engage in the required learning activity?" are worth 
being investigated to identify the most appropriate 
media. 

Educators and trainers throughout the world are trying to 
choose appropriate media for particular instructional 
situations. Today's emphasis on effectiveness (cost or 
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learning outcome) make the role of media increasingly 
important to the total instructional program. The needs 
of adult education and officer training provided an 
excellent opportunity to study media selection for those 
courseware development 

Expert System Development 

The value of the subset of AI, known as expert 
systems, is that it can be implemented on desk-top 
microcomputers and may play a significant role in 
instruction and training. Expert systems can be defined 
(Feigenbaum, 1984) has defined an expert systen' 
program as "an intelligent computer program that uses 
knowledge and inference procedures to solve problems 
that are difficult enough to require significant human 
expertise for their solution. The knowledge necessary to 
perform at such a level, plus the inference procedures 
used» can be thought of as a model of the expertise of 
the best practitioners of the fields".: 

The construction of the MSES followed a four-stage 
process in order to capture the essential tasks involved 
in building a media selection expert system. 

Stage 1. Problem Identification and Resource 
Assessment: This stage addresses the suitability of the 
proposed problem and whether the developer has the 
resources to be successful with the project Media 
selection was identified as a suitable problem for the 
following reasons. 



First* media selection is complicated and requires a 
high level of human expertise to solve it. Second, the 
problem domain is relatively narrow. Identifying 
appropriate media selection attributes and media to 
include in the expert system were identified as the 
critical issues. This was accomplished by thoroughly 
reviewing the literature describing existing media 
selection models and by consulting with human experts. 
Third, the knowledge of human experts could be 
acquired by survey and could be easily represented in 
numeric form in a knowledge base (Matrix format). 

Stage 2. Programming language or Expert System tool 
selection: This stage determined whether an Expert 
System shell or a programming language was most 
appropriate to support the Expert System. Identifying 
the approach used to develop the expert system requires 
the consideration of factors such as in-house expertise, 
budgetary limitations, time, and problem domain. 
The VP-Expert shell was selected to for the media 
selection expert system on the basis of the the 
following reasons. 

Firsi, . '^-Expert is primarily a rule-based expert system 
tool. Its domain control mechanism is backward- 
chaining with limited forward-chaining and <is induction 
feature that can generate rules from data. The tool's 
capability is suitable to solve the media selection 
problem because of the relatively narrow media 
selection problem domain. 



MSES MEDIA POOL 



1. COMPUTER: 3. PROJECTED STILL VISUALS: 

CBT (Mainframe or PC) Slide 
CD-ROM Overhead Projector 

Computer Interactive Vkjeodisc 
Digital Video Interactive 4. GRAPHIC MATERIALS: 

Workbooks 
Printed Job-Aids 
Visual Display 



Compact Disc Interactive 
Expert System 
Computer Simulation 
Computer Game 
Teletext / VkJeotext 
Intelligent Tutoring System 



5. AUDIO: 
Tape/Cassette/Record 
Teleconference/ Audioconference 



6. TRAINING DEVICE: 
Real Object 
Model 

Training Device 

7. MULTI-MEDIA: 
Slide / Tape 
Print / Picture /Tape 
Mutti Image System 



2. MOTION IMAGE: 
Video Disc (Cassette) 
Motion Picture 
Broadcast Television 
Teleconference / VIdeoconference 



8. Tradition: 
Mediated Interactive Lecture 



Figure 1. MSES Media Pool 
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MSES MEDIA SELECiiON ATTRIBUTES 



1. INSTRUCTIONAL SETTING: 
Effective for MulthSite 
IndividuaRzing Instruction 
Effective for small size group use 
Effective for niedium size group use 
Effective for large size group use 

2. LEARNER CHARACTERISTICS: 
Effective for Older Learners 
Effective for Poor Readers 

3. LEARNING OUTCOMES: 
Effective for Learning Facts 
Effective for Learning Categories 
Effective for Leamir.g Procedures 
Effective for Learning Rules 
Effective for Learning Principles 
Effective for Learning PsychonfK>tor Skills 



4. INSTRUCTIONAL EVENTS: 
Informing Learners about the Cijectives 
Presenting Stimulus Materials 
Supporting Practice to Obtain Mastery 
Providing Feedback 

Assessing Performance 
Effective for Interaction 
Effective for SeK Selection 
Effective for Self Pacing 
Accepting Overt Response 
Showing Time Lapse 
Recording Student Perfonnance 
Information Storage and Retrieval 
Effective for Group Interaction 

5. PHYSICAL FACTORS: 
Visual Requirements 
Aural RequrenDents 
Presenting Motion 
Presenting Color 
Presenting Body Cues 



Figure 2. MSES Media Selection Attributes 

Second, VP Expert allows developers to link together 
many smaller knowledge base, which, if combined in 
one big knowledge base, might enceed memory 
capability. This is made possible because VP-Expert 
supports three kinds of integration: DOS execution 
calls, Sprcadsheet access. Database manipulation. 

Third, VP Expert's input and output features allow users 
to interact easily with the program. In addition, the 
hypertext facility allows the developer to easily create 
on-line help utilises for the users. 

The Media Selection expert system utilized many of the 
facilities available through VP Expert. The program 
was integrated with the PC-Pilot Authoring program, 
PC-Painlbmsh graphics, and Turbo Pascal to improve 
its overall performance. 

Stage 3. Prototype Expert System Development: This 
stage addresses \sr \es relating to knowledge acquisition, 
knowledge representation in the expert system, user 
interface development, explanation of the system's 
reasoning, and management uncertainty. 

After reviewing media selection models and consulting 
with human experts, thirty media were identified and 
categorized into eight categories. The MSES media pool 
is presented in Figure 1. Thirty-one media attributes 



were identified and categorized into five media selection 
categories.The MSES media selection attributes arc 
listed in Figure 2. A survey was constructed to acquire 
knowledge from nineteen media specialists who were in 
cither the U.S. Army training division or academic 
environments. The survey responses were statistically 
analyzed and the knowledge was represented in a matrix 
format with thirty media across thirty-one training 
effectiveness attributes. 

In summary, the MSES knowledge base was gathered 
directly from published research and the experience of 
many media specialists. This expertise has been stored 
concisely in a matrix knowledge base and reasoning in a 
matter of only a few seconds for each and every 
objective in a course. Hypertext files were used to 
present on-line help, and the system uses a mouse and 
keyboard to allow users to interact with the Expert 
System. 

Stage 4. Testing, Evaluation, and Maintenance: The 
final stage of the development cf the media selection 
expert system involved estimating the quality of the 
ES. This involves a system evaluation to determine the 
level of expertise and accuracy that can be expected from 
the MSES. There were three different approaches 
followed for the system evaluation: verification, 
v alidation, aiid user acceptance. Human media selection 
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experts and typical users participated in the evaluation 
of the MSES. 

Verification refers to determining whether or not the 
knowledge base and inference engine that drives the 
system is correct. Verification primarily requires the 
domain experts to determine the correctness of the 
knowledge base because inference engines are thought 
tested by the manufacturer. The verification data 
analyses demonstrated agreement across the 19 human 
experts in their understanding of the relationship 
between the media and the media selection attributes. 

Validation refers to determining if the system's advice is 
adequate and at an acceptable level of accuracy. Six 
instructional objectives were developed and were 
categorized according to Gagne's categories of learning 
outcomes. For six instructional objectives, media 
recommendations generated by MSES were compared to 
media recommendations imduced by media specialists 
and target users. Subjective judgements were used to 
determine the quality of the advice given by the system. 
A judgement group, consisting of five experts, was used 
to evaluate the qiiality of the recommendations of the 
media specialists, target users, and MSES. The 
conclusions of MSES validation are presented in the 
following: 

1 . lastructional designers without the MSES have 
a .62 acceptance value. 

2. Instructional designers with the MSES have a 
.7 1 accqHance value. 

3. Media specialists working alone have about a 
.67 accq)tance value. 

4. Media specialists working with MSES have 
about a .71 acceptance value. 

5. Instructional designers performance is 
improved .15 by using the MSES !!! 

6. The MSES knowledge base was gathered 
directly from media selection models ,current 
published research and from the experience of 
nineteen responding media specialists. MSES 
provides the instructional designer an 
inexpensive consultant whose consistent, 
automated decisions and recommendations had 
shown significant improvement on 
instructional designers* decision making. 

User acceptance was included in the evaluation because 
some expert system applications are not used in spite of 
the fact that they function properly. This occurs when 
the target users do not like using the system. Hypertext, 
pull down menus, on-line help facilities, and a tutorial 
were built into MSES to help insure that users would 
find the system friendly to use. 

A user acceptance questionnaire was provided to the 
target user and media specialist after they worked 
through the MSES. The user acceptance results are 
presented in Figure 3. 



Overview of MSES 

The MSES consists of three major modules: the 
Demonstration Module, the Editors and the Media 
Analysis Module. The Demonstration Module is wriucn 
in PC-PILOT and PC-Paint brush was used to create all 
the graphics. The Editor is written in PC-Pilot; utilities 
programs in the advanced features library were also used. 
The Media Analysis Module was developed by using 
VP-Expert shell. During the consultation process, 
external Turbo Pascal programs were called in to handle 
the computation and data input/'output task. The stages 
of MSES reasoning process are shown in Figure 4. 

When consulting with the MSES, users are first asked 
to input instructional objectives; this is represented in 
the first block of Figure 4, Input Training Objectives. 
In the second and third blocks, users view a list of the 
media and the media selection attributes that are 
referenced in the MSES. If users are not familiar with 
any of the entries, they can access hypertext files to 
obtain detailed descriptions of specific media and media 
attributes. 

In the next phase of the program, MSES requires users 
to assign weighting factors to each media selection 
attribute acccHding to their relative importance to their 
instructional objective. This feature is built into the 
program because the only the MSES users can 
understand their audience and local training 
environment. Thus MSES makes media selection 
deeisions based on both human experts* knowledge, and 
users' judgements. 

To insure useable recommendations, the program 
requires us^ to identify those equipment and facilities 
that are available for instruction. Refer to Blocks 5 and 
6 in Figure 4. The MSES produces media 
recommendations based on the complete set of media 
available to users and on the basis of the media pool 
that the user has immediately available. The first set of 
recommendations is provided to allow users information 
so that they can justify obtaining the most appropriate 
media for a learning objective. The second set is 
provided to give the user practical information when 
designing instructional packages. 

The MSES is designed to allow users to identify media 
for multiple training objectives. This process is 
streamlined by having the program record the media that 
the user indicates are available for the first learning 
objective. Refer to Figure 4, Block 5. This results in 
the MSES reasoning process being much simpler for 
additional objectives. Users only need to input a new set 
of weight factors for each additional learning objective. 
The MSES reasoning process for additional objectives 
is represented in Figure 5. 
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Figure 3. MSES user acceptance. 
Conclusion 

Because the evaluation of expert system is expensive 
and time consuming, most of the expert systems 
available today are not evaluated Formal validanon. 
verification and user acceptance has been conducted on 
the MSES. The results demonstrate that the MSES is 



easy to use. the hypertext files arc helpful, includes 
important media and media selection attributes, and 
provides high quality recommendations. Feedback from 
future users will be incorporated into new versions of 
the system. 
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Stages of MSES reasoning process 



This flowchart summarizes all stages of the MSES consultation process. 
Each stage will be described later. 



I 



INPUT 




DESCRIBE 




DESCRIBE MEDIA 


TRAINING 




MEDIA 




SELECTION 


OBJECTIVES 




POOL 




ATTRIBUTES 
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Figure 4. Stages of MSES reasoning process. 



MSES Reasoning Process 



Consultations for additional objectives only require that 
the user enter weight factors. 
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introduction 

This paper is intended to provide the instructional 
technologist having some understanding of expert 
systems with further insight into two of the many steps 
involved in the design and production of such systems: 
KNOWLEDGE ACQUISITION and KNOWLEDGE 
STRUCTURING (also known as KNOWLEDGE 
REPRESENTATION). It is also intended to help 
instructional technologists to see how they might fit 
themselves into the knowledge engineering (expert 
systems building) paradigm and to help knowledge 
engineers to identify domains of instructional 
development research and practice from which they 
could benefit. 

We will begin by examining a generalized model for the 
design and production of an expert system. We wiU then 
look in more detail at the first two steps in this model, 
knowledge acquisition and knowledge representation 
(structuring). The relationship between the instructional 
development process and the knowledge engineering 
process will be emphasized. Finally, characteristics of 
the ideal knowledge acquisition engineer will be 
compared to those of the ideal instructional developer. It 
is hoped that these points will show that the two 
processes are very similar, and that a cross-fertilization 
of both ideas, techniques and people between the two 
fields will prove both possible and profitable. 

The Model 

The expert systems design and production model of 
interest to us is shown in Figure 1 (adapted from Martin 
and Oxman, 1988). As this figure shows, the first task 
involved in designing an expert system is to select an 
apprc^nate problem to solve. This is possibly the 



hardest task to accomplish properly for the knowledge 
engineer. A great many of the jobs we perform every 
day as instructional technologists are currently 
unsuitable for expert system development. (Of course, 
as we develop better technology, it is exactly these 
kinds of fuzzy problems that will eventually provide us 
with the greatest benefits, once we have developed an 
adequate way to deal with thein.) 

For example, any task which is based upon hard-to- 
define knowledge, common sense, or intuition is 
cuTT'tntly unsuitable, given the existing state of the 
technology of knowledge representation and expert 
system design. So is any task in which it is unclear 
whether, when completed, the potential user would ever 
in fact use the expert system. Many oth^ characteristics 
must be met by the ideal task if it is to be suitable (cf., 
Martin and Oxman, 1988, p.l30). 

Once a problem has been selected, the next step is to 
obtain the data-base or set of knowledge that is the 
property of an expert working with this problem. There 
are several aspects to this step, and these will b^^ 
discussed later in the paper. It is sufficient here to noi.. 
that this step is probably the most time consuming and 
expensive step in the design and production process. It 
involves locating a suitable expert (called subject-matter 
expert, or SME, in instructional development, and a 
domain expert in knowledge engineering), eliciting from 
the expert the required knowledge, understandings, 
judgmental procedures and hunches, writing this 
knowledge in a logical and explicit form, and verifying 
the correctness of this written structure with the expert 
(Texas Instruments Teleconference on Expert Systems, 
1987). 
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Figure 1 The Expert Systems Building Model 
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The third step of interest to us is that of knowledge 
structuring, or knowledge representation. This task 
involves converting the acquired and verified knowledge 
into a structure the computer can use and which is best 
designed to enable the user to employ the resulting 
computerized system. This step, also, is composed of 
several parts. The knowledge must be structured in such 
a way that the original relationships among facts, 
descriptions, procedures and probability spaces are 
maintained. It must also be structured in such a way 
that it can be used by the inference engine (that part of 
the expert sy.nen^ that makes decisions, inferences, and 
conducts other reasoning exercises). It must be 
structured in such a way that the factual aspects of the 
knowledge structure can be stored in a data base, and in 
such a way that efficient and convenient search-and- 
reU'ieval sU'ategies can keep the inference engine 
supplied with whatever data it needs to function 
properly and rapidly; and it must be suiictured in such a 
way that a user interface can be conslrucied to the expert 
system enabling the user to easily ask answerable 
questions based upon the acquired expert knowledge. 
Many structuring schemes have been developed, and 
which one is best to use for any given problem is a 
factor of the knowledge domain, the type of questions to 
be asked by the user, the structure of the inference 
engine, and the expert language or shell to be used. The 
resemblance of this process to task analysis, media 
selection and other knowledge representation aspects to 
the instructional design process will be discussed later 
in this paper (Martin and Oxman, 1988; Waterman, 
1986; Benchimol, Levine and Pomerol, 1987; and 
Taylor, 1988). 

The rest of the design and production process for expert 
systems is much more closely related to the computer 
programming profession than to the instructional 
development profession, and so in Figure 1 all steps 
subsequent to knowledge representation have been 
lumped into one, prototyping, with one exception, 
prototype testing. Prototype testing is another step in 
the process that should sound familiar to the 
insu-uctional developer. It is evaluation, including 
summafive, formative and process, and both obtrusive 
and non-obtrusive, both normative and criterion- 
referenced, all thrown into a single pot and stirred well. 
Again, we will discuss this below. 

One area in which the instructional developer and the 
designer of expert systems can help each other, but 
which is not represented in the model given here, is in 
the design of the user interface. The user interface is that 
part of the computer system that includes the displays 
shown to the user, determines what inputs and other 
responses the user is allowed to make, and what 
responses the computer system can return to the user. In 
the case of expert systems, as with good CBI systems, 
the user interface also includes (and is based upon) a 
model of the user. In the case of the expert system, this 
model may be an intelligent system in itself (Systems 
Group, The Open University, 1987); a special language 
interface, known as a natural or quasi-natural language, 



may be employed, and this is in itself a kind of artificial 
intelligence system (Martin and Oxman, 1988, p.226). 

This, then, is the general model of how an expert 
system is designed and produced. It is an overview, and 
much of importance has been omitted. But it should be 
sufficient to show those areas in which insU'uctional 
developers can play a part, based upon a similarity 
between what they normally do and what must be done 
to produce an expert system. Let us now examine some 
of these similarities in more detail, beginning with the 
problem selection step. 



Problem Selection for 
Expert System Development 

The selection of an appropriate problem for use in 
designing an expert system is extremely crucial to 
project success. However, this is not easily 
accomplished. Four issues are involved: identification of 
the task to be accomplished, identification of the people 
involved in producing the system, identification of the 
final objectives and users, and identification of both 
required and available resources (Benchimol, Levine and 
Pomerol, 1987, p. 152). 

Usually a general type of problem is easily selected, in 
terms of either its research value or in terms of its 
practical value within the organization proposing the 
project. But this problem must be narrowed and defined, 
and it must be understood how the project based upon 
the problem might proceed. The problem to be solved 
must be examined in detail in order to produce a high- 
level design for the expert system that is going to solve 
the problem. The refining of the task and the definition 
of the expert system in terms of that task must often 
occur within a vacuum of k.iowledge about either the 
problem or the resulting system, much as is the case in 
the front-end definition of a job or task analysis which, 
eventually, will lead to some instruction or a simulator 
design. Often, for example within the military 
procurement system, the simulator and training system 
must be designed long before the weapon system it 
supports has even been prototyped. The selection of a 
problem and the conceptualization of its solution often 
occur within a similar atmosphere. 

One area of critical importance is the selection of the 
expert. An expert must be available to provide the 
knowledge upon which the expert system is to be based, 
or no expert system can be produced. The first aspect of 
this problem is to determine the level of expertise 
needed. Not all (in fact, not most) expert systems are 
intended to function on the leading edge of a field, 
performing state-of-the-art work. Most are intended to 
aid or replace workers having a low to moderate level of 
expertise, or to help entry-level workers. Some are 
intended to perform routine aspects of a complicated 
multi-step procedure, or to relieve experts of routine 
aspects of their work. A state-of-the-art level of 
expertise is not needed for this sort of task. More 
expertise is needed on the part of the human expert 
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supplying the developer with the knowledge base than 
is intended to be incorporated into the program, but a 
world-renown expert is not needed to provide the 
knowledge base for advising an entry-level clerk. 
Further, on those projects in which top-level expertise 
is needed, some of the demand for the expert's time can 
be relieved by providing a lower-level expert for use 
during the initial efforts. Thus, the determination of the 
type and level of expertise needed is very important. 
There seems to be no model or theory of how to do this 
at the present time. Some work is being done on 
defining the nature of expertise and of the expert (Kart, 
1986). Most instructional development projects involve 
the identification of the subject-matter expertise needed, 
both in terms of quantity and of level of expertise 
represented, as a part of the original project plan. A 
theoretical foundation for doing this within the 
instructional development world has also not been 
developed, but most instructional developers do this 
intuitively fairly well. Further, a couple of approaches 
of a procedural nature have been developed 

One of these methods for defining the required level of 
expertise is to define mastery. This is done by analyzing 
the tasks to be performed in the ar^a under 
consideration, and then preparing behavioral objectives 
for these tasks. Criteria defining an adequate 
performance for each of the objectives are then 
established. The sum total of these objectives, within 
the limits set by the performance criteria, defines the 
expertise needed. This expertise is defined directly in 
terms of the behaviors the expert must be capable of, 
and indircctly in terms of the knowledge the expert mu'^t 
have in order to perform these behaviors. How does one 
judge v'lether or not the set of objectives covers an 
adequate amount of expertise, and how does one know if 
the criteria are adequate to ensure expert performance at 
an acceptable level? By applying the 80/20 rule. This 
rule states, as a rule of thumb, that the knowledge base 
established with 20% of the content required to establish 
the complete knowledge base will he- adequate to solve 
80% of the problems the expert sysiem will encounter. 
This rule of thumb is not always accurate, but it 
generally worics. To judge the adequacy of coverage of 
the objectives and their criteria^ examine the problems 
in an actual real-world environment, and classify them 
into those the objectives and criteria adequately handle 
and those that they do not handle satisfactorily. When 
80% of these problems are adequately handled by your 
set of objectives, you have established an adequate 
definition of the required expertise, both in terms of 
breadth and of depth. The methods currently being used 
by instructional developers can undoubtedly be 
improved upon through the development of a better 
theoretical understanding of the nature of expertise, and 
of better procedures based upon this improved 
understanding. This area should prove to be a fruitful 
area of theory building for instructional developers. 

The users of the system must be identified at this stage, 
and again, instructional developers have a long history 
of dealing with this problem, both successfully and 



unsuccessfully, as also have evaluation specialists. Who 
are the real clients for the products resulting from a 
project, and are they the same people who are the clients 
for the development process? Are they the same people 
who are paying for the project? Are they represented in 
the design process? Can the project succeed if they are 
ignored? Here, again, is an area in which both 
instructional development and knowledge engineering 
can benefit through the explication of the processes 
normally used in the instructional development and 
evaluation fields. 

Instructional developers are familiar with the process of 
resource identification as a part of the initial planning 
phase of a project, and the same is true of the 
knowledge engineering project. Before the problem is 
finally settled upon, it must be clearly understood what 
resources are going to be needed, and from where they 
are to come. Standard resource allocation and project 
scheduling techniques are applicable here, but the 
peculiarities of the expert system development process 
must be kept in mind during their application 
(Benchimol, Levine and Pomerol, p. 164), More 
published case studies of this phase of expert system 
development projects are needed, especially in detail 
great enough to show how standard processes can be 
applied, and how they must be modified in order to 
serve the expert systems project. 

Knowledge Elicltatlon: 
Human Relations Aspects 

Knowledge elicitation and knowledge structuring or 
representation are often considered to be one task by 
writers on expert systems from the computer world. 
One reason fc this is that they are often performed by 
the same people, and another is that they are usually 
performed simultaneously, A third reason, we suspect, 
is that the typical knowledge engineer knows far more 
about knowledge representation than about knowledge 
elicitation, at /*east in a formal or structured sense. 

One characteristic shared in common between the 
instructional developer and the knowledge engineer is 
the need to conserve the expert's time during knowledge 
elicitation. The least amount of time the knowledge 
engineer requires of the expert, the cheaper the project 
cost will be and the faster the project will be completed 
(Martin and Oxman, 1988, p.434). However, very little 
knowledge exists concerning how to minimize the time 
required of the expert The knowledge engineer should 
become generally familiar with the field before 
approaching the expert, through the reading of text 
books and other materials relating to the project And he 
must be a good listener, quick thinker, and a rapid 
learner. These are also requirements for the good 
instructional developer. However, these are minimal 
requirements for optimization of the expert's time. 
Much more work needs to be performed concerning 
making optimal usage of the expert's time. 
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The knowledge clicilor must also maintain the expert's 
interest in the project, make sure he understands the role 
he is to play, make sure he understands the level of 
knowledge desired, and make sui^-^ his good will towards 
the project and towards the Knowledge elicitor is 
maintained. If a failure in any of these things occurs, an 
accurate diagnosis of the prcblem is absolutely 
necessary if the damage is to be repaired. Again, within 
the realm of knowledge engineering these aspects of 
knowledge elicilation have not been formalized in any 
liicory, and are usually dismissed after a short discussion 
summarizing tlieir importance to project success and 
then stating that they call for the knowledge engineer to 
exhibit good communications and personal relations 
skills. \Vhi!e this is certainly true, it is not enough to 
be of any specific help in conducting a specific project 

Several lines of research conducted in instructional 
development can be of use in dealing with these 
problems, and should be extended (both within the 
instructional development Held and as applied to the 
expert system development process). Research on client- 
consultant interactions, initial meetings, appropriate 
model, and role changes at different points in the 
consulting process, have been conducted, and some 
efforts have been devoted to teaching students to apply 
the findings of this research (Rutt, 1984; Price, 1984; 
Coscarelli and Stonewater, 1984). Further, preliminary 
work by Gentry and Trimbly on the formal analysis of 
communication interfaces for instructional developmf*nt 
projects (Gentry and Trimbly, 1984) certainly has 
applicability here. A recent review of the interpersonal 
skills required by instructional developers has been 
prepared by Curtis (1990). Work in describing and 
prescribing the consultant relationship has also been 
conducted in other fields, such as medicine and 
personnel psychology (i.e., Johnson, 1978; 
Schcin,l978; and Tilles, 1961). Again, these studies 
need to be followed up, and ways in which the 
knowledge elicitation process i-; similar to and different 
from the models researched in the cited studies needs to 
be explicated. The methodologies for conducting and 
applying this research are already to be found in the 
instructional development literature. 

Knowledge Elicitation: Methodologies 

The most commonly-discussed aspect of knowledge 
elicitation within the knowledge engineering literature 
is th'^t dealing with formal approaches to the elicitation 
of knowledge from the expert. Robert Pearson (Pearson, 
1988) provides a good summary of the most 
commonly-discussed methods. These are the structured 
interview, the use of probability estimation, machine 
induction and the Repertory Grid. Other methods 
discussed in the literature include the use of historical 
records, observation, both obtrusive and non-obtrusive, 
thought experiment<^ and case sUidics CMartin and 
Uxinan, 1988, p. 175). 

The general methods described in these sources are for 
the most part those methods the instructional designer 



is already familiar with and to some extent an expert in. 
Some exceptions might be \iiC Repertory Grid, machine 
induction, and probability estimation. Pearson also 
discusses soniC aspects of the structured interview that 
may not be familiar within instructional development 

The Repertory Grid is a "tool used by psychologists to 
represent a person's view of a problem in terms of 
elements and constructs" (Hart, 1986, p. 174). The grid 
is a two-dimensional table (characteristics X cases). The 
characteristics are bipolar, such as strong and weak. The 
cases are ratcxi on each characteristic on a scale of 1 to 5. 
Instructional developers have used such matrices in 
other contexts, such as with the semantic differential, 
attitude measurement, and in attribute learning theory 
research. Once such a matrix has been developed by the 
expert, it can be analyzed for patterns, either through 
inspection or through more sophisticated methods, such 
as factor analysis, cluster analysis, or profile matching 
(Pearson, 1988; Hart, 1986). Further, since the world 
view of each expert is likely to be different from that of 
any other expert, it is often profitable to compare the 
grids of different experts. 

Machine induction is a process in which the computer 
generates generalizations when given specific examples 
(called training sets) as inputs. Special software, called 
inductive algorithms, has been developed to carry ou* 
this process. As Pearson points out (Pearson, 1988), 
the results are only as good as the examples chosen for 
inclusion in the training set. Further, this method can 
only be used with knowledge to the extent that it can be 
rcduceu to numerical representations. 

The development of a probablistic kn-vwledge 
representation is based upon the assumption tliat much 
of an expert's knowledge is of a non-exact nature. Thus 
he is asked the degree of certainty he attaches to various 
of his decisions and recommendations. These are 
represented within the expert system as weights attached 
to other outputs, and attempt to mimic the expert's 
judgements in these matters (Pcar;>on, 1988). 

To use these methods of knowledge elicitation. the 
knowledge engineer clearly needs an in-depth 
understanding not only of the particular technique being 
used, but also of statistics and mathematics in general. 
Typically instructional developers and evaluation 
experts do not have such an extensive background in 
quantitative science. However, through appropriate 
training, they could be piovided with such a 
background. Not only would this equip them for 
performing knowledge elicitation tasks for the 
development of expert systems, but .vould allow them 
to use the knowledge en/»ineering methods within more 
traditional instructional development projects. 

On the olhci liaiiu, the instructional design and 
development literature is replete with descriptions of 
practical approaches to job and task analysis, which 
have shown themselves to be effective and relatively 
efficient (see Carlisle 1986). Whereas the bulk of these 
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techniques were developed to analyze procedural tasks, 
there are some, such as Critical Incident Anal>sis, 
Problem Analysis, or Case Analysis, which have a 
track record of success m the domain of complex 
heuristic tasks such as management decision making. 
The study of these methodologies may enhance the 
efficiency of knowledge engineers in performing the 
knowledge elicitation stage of expert systems 
development 

Knowledge Elicitation: Methodologies 

In general, the problem of knowledge elicitation faced 
by the knowledge engineer is more difficult than that 
faced in the typical instructional development project 
(although, of course, there are exceptions to this) (Dills, 
1988). This stems from the fact that the level and type 
of knowledge that the expert system must model is 
usually of a much more complex, more fundantental, 
and a less well defined nature than is the knowledge 
typically represented by a CBI system or a textbook. A 
greater similarity exists between the two types of 
knowledge when the instructional developer is 
attempting to mold the student's attitudes, represent 
non-verbal behavior for the student U> model, or to teach 
in interactive process through the shaping of behavior 
during interactive practice sessions. It is in developing a 
knowledge base for these instructional activities that the 
instructional developer experiences the greatest 
difficulty, and the standard methods for task analysis and 
content analysis are most likely to fail. And these tasks 
are similar to ones the knowledge engineer must face, 
especially in projects in which the expert system is to 
exhibit a high level of expertise. 

The expert system must not only incorporate factual 
knowledge, but also must represent procedures, 
concepts, principles, and standards. But this is only the 
beginning. The expert system must reason in such a 
way as to mimic the expert. It must make judgments 
which are only probably true, it must make them as the 
expert v'ould, and its judgements must have (at least) 
the same probability of bc«:ig correct as the judgements 
made by the expert hin^oclf. 

Further, the expert system usually contains an 
explanation facility. The Explanation Facility tells the 
user, usually upon request, ho v a particular decision, 
recommendation or conclusion was reached by the 
expert system. This includes displaying the reasoning 
pattern used, the data analyzed, and the rules utilized. 
Thus, the expert system must have built into it not 
'"nly sufficient knowledge to duplicate the expert's 
outputs (conclusions, recommendations) but must also 
be able to describe the expert's thought processes and 
reasoning patterns (Mairin and Oxman, 1988, p.28). 

There are projects in which only one expert is used. It 
may be that the level of expertise required is such that 
any expert in the field would do, or that little differences 
arnong experts are unimportant at the level of the 
project. It mL^ also, that only one expert is 



available to the project, or that only one expert exists ai 
the level of expertise required. Normally, however, more 
than one, and in some cases several, experts are utilized. 
Sometimes the experts are equally utilized, as a team. 
More often, they are used to provide expertise about 
different aspects of the problem to be solved. When they 
are utilized to different degrees, sometimes one is titc 
central, or main, exi>ert, and the others are used to 
validate the knowledge base of the main ex])ert, or to 
supplement it. Other times, the main expert is used 
only after the other experis have laid an appropriate 
groundwork, thus conserving the main expert's time. In 
rome cases, unfortunately, the main expert is lost 
sometime after the project has begun, and other exfxjris 
must be utilized to replace him. 

In all of the multiple-expert situations listed above, the 
knowledge elicitation problem is complicated by the 
special problems involved in reconciling and validating 
the experts with one another. The communications and 
human relations problems become much more 
complicated because of the increased number of people 
involved v\d the greater chance for emotional problems 
U> develop. One of the best methods currently used in 
such situations, by the way, is the Repertory Grid, 
especially when coupled with structured interview 
techniques. 

Another aspect of importance in the understanding of 
expertise is to understand the expert is a person - his 
world view, his philosophical bent, his pre-conditioned 
reactions to specific types of situations. The knowledge 
base in the philosophy of science, the psychology of 
the human world-view, and die nature of the scientific 
method that many instructional developers and 
evaluation experts possess, is typically not found 
among professional knowledge engineers. The 
instructional developer can make use of this kiK)wledge 
in trying to establish the reasoning patterns and 
paradigmatic aspects of the expert's understanding. This 
should enable the instructional developer to more 
quickly model the more fundamental aspects of the 
expert than would someone lacking this background 
(see, for example, Davies, 1984; Siahl, 1984). A 
methodology for the geneaation of methodologies was 
developed by Hutchinson and his associates 
(Hutchinson, 1984). This could be adapted to the 
purpose of formalizing the use of the foundational 
knowledge from philosophy ard psychology for the 
development of new methods for eliciting the deep 
structure of the expert's world view concerning the 
problem at hand. Such developments should prove 
fruitful not only to knowled{;e engineering and 
instructional development, but also to cognitive 
psychology and the philosophy of science. 

Yet another area of development is the design of 
software for use in knowledge acquisition. One type of 
software, just beginning to be developed, is the 
knowledge acquisition facility (Olson 1990). This is a 
part of th^ software that carries on a dialogue with the 
expert, eliciting knowledge and procedi" : from him. 
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and directly inserting them into the data base. Such a 
system may be specially designed for the system being 
constructed, or may operate through the user interface. 
The same syslcm may be utilized later to update or 
expand the exfiCrt system as new knowledge is acquired 
and old knowledge becomes outdated (Martin and 
Oxman. 1988). 

A second type of software of use in the knowledge 
elicitation process is a software package that tracks 
pieces of knowledge throughout a computer system and 
other developmental processes as well, and identifies 
inconsistencies within the data base, as well as 
redundancies. One such system, called TRACE, is 
currently under development (Olson 1990). 

New meth<ods for eliciting knowledge are badly needed 
within knowledge engineering. Current methods are 
time consuming, costly, and are not guaranteed to 
properly elicit the needed knowledge without the 
intuitive intervention of an experienced knowledge 
elicitor. As in instructional development (Dills and 
Bass. 1984), much of the knowledge elicitation function 
of knowledge engineering is currently an art form, not a 
technology, and so cannot be automated. Perhaps this 
will always be the case, but at least it should be 
possible to develop better techniques to use when 
performing the art. Instructional developers are in a 
good position to develop some of these, and would 
certainly benefit in turn from any such new techniques 
that might be developed. 

Knowledge Representation 

Knowledge in an expert system is structured, or 
arranged, in two pans, which arc built and stored 
separately, and which are designed to interact with each 
other. One pan is called the inference engine, and 
contains the knowledge concerned with how to reason. 
The other pan is called the knowledge base, and 
contains the facts, rules and other types of b»owledge 
used by the inference engine in reasoning. 

The knowledge engineer must decide what SL jciure to 
use in arranging and organirJng \hc hnc* "ledge b ^ so 
that it can be accessed and utilized by th< ; computer, and 
ultimately by the user, in the most efficient and 
effective manner. To do so, two questions must be 
answered. The first is how the inference engine is lo 
function. The second is how the knowledge is organized 
within the knowledge do.main. Both of these questions 
must be answered in terms of the logic of the field in 
question, and also in terms of the logical structures 
available within the computer. 

Several stnjctures for representing knowledge within the 
knowledge base have been developed, and are usually 
dc<;crihcd in term*; of how various aspects of the expert's 
knowledge relate to each other. Special languages have 
been developed to simplify the implementation of lliese 
structures, and these are likely to be used in any 
significantly large p.oject. They reduce lime and cost for 



constnicting the expcn system, but they also add their 
own restrictions and limitations to the resulting system. 
Therefore, one decision that must be made is which 
language is to be used to structure the knowledge. This 
decision must be made simultaneously with the decision 
concerning which logical structure to use in 
representing the knowledge (cf.. Waterman, 1985, 
Chapters 3, 7 and 8; Martin and Oxman, 1988). 

The instructional developer is not likely to be familiar 
with the ways in which knowledge can be structured 
within a knowledge base. However, he does perform 
tasks which are functionally similar. He structures the 
flow of information within lesson designs when 
producing CBI. If he micro*designs the CBI lesson, he 
stru:tures the knowledge even more carefully. In doing 
this, he undoubtedly tries to discover an internal, natural 
structure to the subject matter being taught, and uses 
this structure in designing his lessons. He also attempts 
to structure the information according to design 
heuristics, and according to what is known concerning 
the psychology of learning. In designing a CBI or 
textbook page, he attempts to make use of the research 
on text design, and is tliercfore applying another logical 
structure to the info^nation. In doing these things, the 
instructional developer is restricted by the CBI 
authoring language, the subject matter structure, and ihn 
needs and abilities of the user, much as is i^iC 
knowledge engineer. 

Four knowledge representation schemes dominate the 
knowledge engineering field, and the instructional 
developer should be familiar with these if he is to work 
with expert systems. These arc rules, frames, semantic 
networks and first-order logic. Others are being 
developed, and eventually may supplant the four. One 
should be aware of these, but currently almost alt work 
is performea using one or more of these cthods 
(Martin and Oxman. 1988). 

Rules are the most commonly-used form. Know Jge is 
represented as a series of logical IF-THEN statements, 
and the inference engine calls these rules in a sequence 
determined by the subject matter, detennines if the IF 
statement has been fulfilleJ, and if so, carries out the 
THEN :>tatement The THEN statement may be to take 
an a tion outside the computer, such as print out a 
decision, or to change an internal variable, such as 
establish the numerical value of a variable in an 
equation. 

Frames and semantic networks are similar, and both 
depend upon groupings. Semantic networks consist of a 
series of nodes connected by lines. When drawn, a 
diagram representing a semantic network looks much 
like a fiow chart for an instructional development 
project, or a PERT chart. The nodes represent concepts, 
and the connecting lines represent relationships. There 
is a hierarchical relationsnip among the concepts, such 
that concepts lower in the diagram inherit the 
characteristics of concepts higher In the sta'v.Mre. Thus 
a kind of umbrella arrangenicnt exists. If 'ship" is a 
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hi jh-level concept, and it is a characteristic defined into 
the system that "ships" have "hulls", then, if "aircraft 
carrier" is a lower-level concept, and is connected to the 
concept "ship" by the "is-one" relationship, the 
infercncc engine will automatically infer that "aircraft 
carriers" have "hulls" (Waicrman, 1986). 

Fran ,s are similar to semantic networks. If the nodes 
are considered frames, then the frames indicate relations 
to other frames. Each frame is a container, and the lines 
in the diagram connect the frames, and indicate frame-to- 
frame relationships. The frame itself can be considered a 
concept The designer then puis far««. procedures, 
concept names, design information Ois the nanne of t!ie 
expert who uses the concept) into :he frame. This 
information is stored in slots. Each frame has a certain 
number of slots, or pigeonholes, and each piece of 
informatioa fills one or more slots. 

Thus each frame is a cluster of related facts, and the 
cluster itself relates the facts to other clusters. The 
frames may cx may not have an umbrella relationship as 
described above for semantic nets (Benchimol, Levine 
andPomcrol, 1987). 

First-order logic as a knowledge-representation scheme 
involves representing the internal structure of the 
knowledge domain using prepositional k)gic. The facts 
and other objects of the domain are rq}resented as 
elements within prc^sitions, and propositions are 
related through the use of propositional logic. Such a 
framework has the advantage that it is a mirror image of 
the kna^edge domain and/or of the expert's deeo mental 
structure. It has the disadvantage that it must be tailor- 
made for each application, at least if it is to be 
maximally efficient and yet preserve its nature as a 
minor image of the expert's mental structure; it does 
not lend itself readily to incorporation into a 
commercially-provided shell (although, when an exact 
mirror image is not needed, a shell can be used to 
provide an approximation to this structure using this 
representational scheme). 

Declarative knowledge can be represented by variotis 
types of logical structures. Commonly used are 
pr(^)OSitional calculus, predicate calculus, first-order 
predicate calculus, and Horn clause logic. Statements 
such as "Lassie is a dog" and "If Lassie is a dog, then 
Lassie is a mammal" would result in the inference 
engine deciding that "Lassie is a mammal" in these 
systems. More concerning these types of representations 
can be learned from Martin and Oxman, chapter 10, or 
Benchimol, Levine and Pomerol, chapters 3,4 and 5. 

Again, most instructional developers do not have a solid 
backgroimd in symbolic logic, and so will have trouble 
with these schemes. However, this situation could 
easily be remedied by a short course in the instructional 
development department or a summer institute if 
enough developers wanted tn work with expert systems. 
And these schemes are easily learned, so developers can 
learn them on their own. If one is to serve as a front-end 



expert on an expert-system project, one must become 
familiar with these schemes in some manner. 

One of the areas in which developers can aid the expert 
system field is to develop ways in which the internal 
structure of a field can be easily and quickly determined, 
so titat this information can be used in determining how 
the knowledge is to be represented. Again, the suucture 
of the field can probably always be deduced from 
information concerning the paradigmatic and other 
world-view features of the expert's knowledge structure. 
As argued earlier, a formalized method for determining 
the paradigm of the expert and of translating this into a 
knowledge structure could be developed, and would be 
very useful. Further, most knowledge engineers have no 
background in this area, and many evaluators and 
instructional developers do. 

The structure of the inference engine must also be 
determined* and several structures (in this case, 
procedures for reaching a decision) are in conunon use. 
(Xten a ready-made inference engine is purchased, rather 
than ctistom-made, but again, its workings must be 
tmderstood if an appropriate purchase is to be made. 
Some of t'»e most common inference scheiues are 
(Martin and Oxman, 1988, p.64): 



backward chaining 
forward chaining 
breadth-first search 
depth-first search 
heuristic search 



probtem reduction 
pattern matching 
unification 
event-driven control 
hierarchical control 



These methods of inference will not be described here, 
since they are fully discussed in the references. But the 
instructional developer must be familiar with each of 
thein, so that the proper inference technique can be 
matched to the problem at harxl, and the knowledge base 
can be represented so as to properly fit the inference 
engine. The knowledge elicitor must be familiar with 
the ways of building an inference engine and a 
knowledge base, since the way the knowledge is 
structured and used must be decided as the knowledge is 
being acquired. Typically an expert system is being 
prototyped as the knowledge structure is being 
developed, and early decisions are both necessary and 
critical to future expansions. Thus, the instructional 
developer, if he is to utilize his knowledge of front-end 
analysis in knowledge acquisition for expert systems 
development, must become knowledgeable concerning 
how this knowledge is to be structured and utilized by 
the computer system. 

Thus the instructional develq)er is required lo develop a 
knowledge of inference engines and knowledge 
representation schemes, and this requires that he develc^ 
a knowledge of Ipgic, and of various inference schemes. 
It does not require him to become a computer 
programmer or a systems engineer. It does require that 
he become familiar with the languages typically used in 
expert system work, but on a conceptual level. In most 
projects, the programming will be performed by a 



ERLC 



181 



9:^ 



Dills and Romiszowski 



specialist But the developer must communicate wiih 
this specialist, and must know the restrictions and 
advantages placed upon him by the language being used. 

Testing 

Not only must the resulting computer programs be 
tested* but the resulting knowledge structure and 
inference pauems must «.!so be tested. Further, the 
usefulness o[ the resulting product to users, and the need 
for updated knowledge, must be evaluated. These things 
must not only happen when the system has been 
prototyped, but at each step in its evolution, and 
periodically after it has been completed and turned over 
to the customer. It is clear that instructional developers 
possess expertise of great value here, especially in 
designing and conducting user interviews, evaluating 
aging knowledge bases, and verifying system-made 
decisions against the real worid. 

User Interfaces 

Instructional developers possess knowledge and 
experience in two fields that are related to the design of 
the user interface of expert systems. The first concerns 
screen design and human factors work. The design of the 
interface for the user is similar to the design of an 
interface for a CBI system, and furthermore, involves a 
great deal of screen and text design considerations. 
Further, ihe functional design of the interface will 
determine what the user can ask the system, the type 
and manner in which he gives the system data* and the 
type of reply he can receive. Instructional developers 
already have standard ways of discovering the 
appropriate forms for such interactions in terms of the 
user's intentions. 

A second area in which instructional developers can 
make a contribution to the interface design is through 
their knowledge of cognitive styles. It is known that 
not all experts think in the same manner, and that, 
further, experts and itovices in a field think differendy. 
The expert system must reason as the expert does, at 
least in high-level exert systems. But if the user were an 
expert himself, and reasoned as the expert, he probably 
would not be using the system at all. This is not 
always true, of course, but often it is. Therefore, when 
the novice or the other user receives an answer to a 
question, and asks the Explanation Facility to explain 
how the answer was reached, it would be useful if this 
answer could be given both in terms of how the real 
expert would arrive at this answer (supposedly, this is 
also the way the computer arrived at the answer), and 
also in terms of how this answer could be arrived at 
following the reasoning style of the user. In other 
words, if the expert is a "rcasoner by comparisons", and 
arrived at the answer by comparing various factors, this 
solution would be explained. The computer could 
u-anslate this into the user's style, which might be 
"reasoning by contrasts", and present this chain of 
inference to the user also, once it learned what his own 
style was. 



Currently, no such system exists, but by modeling the 
student or other user, such a system could be developed 
and attached to almost any independently-developed 
expert system. Such a system would be particularly 
useful in training people to reason according to different 
styles* or in using expert systems as instructional 
delivery devices. Instructional developers are in i»ime 
positions to design such a student modeVexplanation 
provider. 

Summary 

It is argued that instructional developers are in an 
excellent position to make contributions to the field of 
expert systems develc^mient, particularly in the areas of 
knowledge acquisition, knowledge structuring, testings 
and designing user interfaces. To do this, they must 
become familiar with the tools, systems and ai^>roaches 
used in the field by knowledge engineers if they are to 
make this contribution. 

It is further argued that knowledge engineers and other 
people from the computer field who are involved in 
knowtedge acquisition and structuring tasks can benefit 
from studying the techniques and sy^proaches used in 
task analysis* needs analysis and similar enterprises. 
They can also benefit by becoming more sensitive to 
the nuances of meaning and mental structures of the 
expert being modeled* and one way to do this is through 
a study of instructional development, especially as this 
field incorporates knowledge from such areas as the 
philosophy of science and world- view psychology. 

Once they have learned the fundamentals of the field, 
they can apply their specialized knowledge and 
experience both as practitioners and as researchers. In 
doing so* they will open up a new technology to 
educational use, as well as provide the knowledge 
engineer with new tools that he will find quite useful. 

References 

Benchimol, G.; Levine, P.; and Pomerol, J.. (1987). 

Developing Expert Systems for Business. London: 

Kogan Page/North Academic Press. 
Cariisle, K. (1986). Analyzing Jobs and Tasks. 

Englewood Cliffs, NJ: Educational Technology 

Publications. 

Coscarelli, W. & Stonewater, J. (1984). Psychological 
Typologies and the Dynamics of Consultant 
Relationships. In R. Bass and C. Dills. (Eds.) 
instructional Development: The State of the Art, 
Vol. II., DuBuque, lA: Kendall Hunt Publishing 
Co. 

Curtis, R. & Nestor, D. (1990). Interpersonal Skills: 
Critical Attributes for Instructional Developers. 
instructional Developments, 7(2), 17-19. 

Davies, I.. (1984)., Instructional Development: 
Themata, Archetypes Paradigms and Models. In R. 
Bass and C. Dills. (Eds.) instructional 
Development: The State of the Art, Vol. II., 
DuBuque, lA: Kendall Hunt Publishing Co. 



ERLC 



182 



m 



Dills, C (1988). Expertise, Knowledge Acquisition and 
Knowledge Structuring. F^)er delivered to Syracuse 
University School of Education, Department of 
Instructional Design, Development and Evaluation 
Professional Development Summer Institute; 
Syracuse, New York. 

Dills. C. & Bass, R. (1984). Insuuctional 
Development: Art? Science? Both?" In R. Bass and 

C. Dills. (Eds.) Instructional Development: The 
State of the Art, Vol. II., DuBuque, lA: Kendall 
Hunt Publishing Co. 

Gentry, C. & Trimby, M. (1984). Interface Analysis of 
ID Systems. In R. Bass and C. Dills. (Eds.) 
Instructional Development: The State of the Art, 
Vol. II., DuBuque, lA: Kendall Hunt Publishing 
Co. 

Hart, A. (1986). Knowledge Acquisition for Expert 
Systems. London: Kogan Page. 

Hutchinson, T. .(1984). Metamethodology. In R. Bass 
and C. Dills. Eds.) Instructional Development: The 
State of the Art, Vol. 11., DuBuque, lA: Kendall 
Hunt Publishing Co. 

Johnson, R. (1978). Individual Styles of Decision- 
Making: A Theoretical Model for Counselling. The 
Personnel and Guidance Journal. 56(9), 530-536. 

Martin. J. A Oxman, S. (1988). Building Expert 
Systems: A Tutorial. Englewood Cliffs. NJ: 
PtenticeHall. 

Olson. M. (August, 1990). Task Analysis Workstation 
as a Front-End to Knowledge Acquisition for 
Expert Systems: The TRACE System". A paper 
delivered at the Sixth Annual Conference On 
Applications of Artificial Intelligence and CD- 
ROM in Education and Training; Washington. 

D. C. 

Pearson. R. (1988). Knowledge Elicitation: A Study 
Guide. In A. Romiszowski and R. Young. (Eds.). 



Expert Systems and Media Selection 



Expert Systems in Education and Training. 

University School of Education. Department of 

Instructional Design, Development and Evaluation 

Professional Development Summer Institute; 

Syracuse. New YoriL 
Price. R. (1984). The Initial Client Conference: 

Implications for the Continuing Relationship.** In 

R. Bass and C. Dills. (Eds.) Instructional 

Development: The State of the Art, Vol. II.. 

DuBuque. lA: Kendall Hunt Publishing Co. 
Rutt. D. (1984). Consultation in Insuuctional 

Development: A First Look. In R. Bass and C. 

Dills. (Eds.) Instructional Development: The State 

of the Art, Vol. II.. DuBuque. lA: Kendall Hunt 

Publishing Co. 
Schein. E. (1969). Process Consultation: Its Role In 

Organizational Development. Reading. MA: 

Addison Wesley. 
Slahl. R. (1984). Cognitive Theory Within the 

Framework of an Information Processing Model 

and Learning Hierarchy: A Viable Alternative to the 

Bloom-Mager System. In R. Bass and C. Dills. 

(Eds.) Instructional Development: The State of the 

Art, Vol. IL. DuBuque. lA: Kendal! Hunt 

Publishing Co. 
The Systems Group. London University, the UK. 

(1987). Expert Systems. London: Manpower 

Services Commission. 
Taylor. W.. (1988). What Every Engineer Should Know 

About AI. C:ambridge. MA: The MIT Press; 
Texas Insuuments. (1987). Third Annual Texas 

Instruments Teleconference On Expert Systems. 

[Video Tape]. 

Tilles. S. (1%1). Understanding the Consultant's Role. 

Harvard Business Review, 39, 87-89. 
Waterman. D. (1985). A Guide to Expert Systems. 

Reading, MA: Addison Wesley. 




183 
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OR LEARNING BY DOING 



G. Ilolweg, ondF. Huber, Institutes for Information Processing Graz (IICM) 



Abstract 

We present an extension to authoring systems for 
tutorial CAI. It enables authors to create lessons which 
arc highly interactive: students can manipulate scenes, 
click on buttons, change sliders, etc. The extension 
improves understanding by shifting focus from passive 
reception of material to active interaction and learning- 
by-doing. The purposed system requir^*^ no 
programming at all, but relies on a graphics, user 
interface for authors of tutorial material. 

Motivation 

In recent years there has been increasing interest in 
computer-assisted instruction. With the advent of cheap, 
but powerful workstations one of the major drawbacks 
of former CAI attempts, namely high costs, has 
disappeared 

We nevertheless believe that CAI has to change 
drastically to be applied universally. Current tutorial 
material lacks an important aspect' user interaction. Our 
experience as teachers has shown that many students 
faithfully believe thai they understand what they just 
read or heard. But when it comes to applying this 
knowledge it soon turns out that there is a gap between 
understanding something and actually applying it. Only 
when you actually have to do something you really start 
to understand it (see, for example, Ehrmann, 1987). 

There have been of course many attempts to overcome 
the passive role of students in typical tutorial lessons. 
Besides question-answer dialogs, which we won't cover 
here, since they are well established (see, for example 
HyperTRAIN (HyperTrain, 1989), cT (Sherwood, 
1988a; Sherwood, 1988b) or TenCORE (TenCORE, 
1985; Klass, 1984)], much effort has gone into 
simulations. There students can woric in microworlds, 
manipulating parameters of a model, changing 
ingredients in a chemical experiment, etc. Simulations 
have their justification, but they still form their own 
niche of the broad spectrum of CAI: they are more 
interactive than plain tutorial material, but it is still the 
program behind the system, which actually does the 
necessary calculations or applies a certain strategy. 

However, even simulations come short of highly 
interactive teaching. At the present time, a number of 
systems enable the student to have a more active role, 
especially in intelligent tutoring systems. Here, a 
student can sometimes take over the initiative and ask 
questions or request explanations. It seems that the 
effort to create such systems is far beyond what even a 
small team can accomplish. What we propose here is a 



system somewhere between simulations and inteUigent 
tutoring: the student should have more means of 
interacting witli the system, and should have a more 
active role which gives him better insighL 

Ut us look at some examples first, before we explain 
what the system looks like from the author's point of 
view. The first example is from computer science. Let 
us assiune that the student is working through a lesson 
on heapsort (if you do not know how heapsort works, 
do not let it worry you here). Like all sorting 
algorithms heapsort has to exchange some numbers to 
get them into the right order. Each sorting algorithm 
has its own way of determining what numbers to 
exchange next. It is essential to understand how 
heapsort rearranges the numbers to be able to write a 
program that utilizes heapsort, or to analyze the amount 
of work necessary to sort n numbers. 

Hc£^sort uses a binary tree for sorting of numbers. To 
test whether students understand how heapsort 
determines two numbers that have to be exchanged the 
students could be asked to do it on their own: they 
move the cursor to the position they think is right, drag 
the number away a little bit, take the other number and 
move it to the original position of the first number. 
Finally, drag the first number to the empty place where 
the second number was displayed before. A^uming that 
the students exchanged the correct numbers, they can 
now continue working. If the students chose the wrong 
numbers, the system would give them some feedback, 
redraw the original scene and ask them for another try. 

The second example is taken from a course for mechanic 
apprentices. Let us assume that the students see a 
picture of an engine on the screen, and they are asked to 
take it apart. There is a certain order to do this. Instead 
of showing the students a multiple choice question (or 
similar restricted textual interaction) the students can 
take the engine apart by simply clicking on parts that 
have to be removed. The system keeps track of what has 
already been done to ensure that, say, a cover cannot be 
removed before all bolls have been removed. 

Both examples show what we have in mind: students 
should have a more active role and actually do things 
instead of just describing them. We believe that a 
simple set of "interactors" will make it possible to 
create interactive pieces such as the two situations 
described above without any programn;ing. How tliis 
could be done is described in the next chapter. 
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A Hierarchy of Interactors 

An interactor is an object on the screen that reacts to 
input from the user in a certain way. We propose an 
object-oriented hierarchy of interactors which is shown 
in Figure 1. The hierarchy is similar to recent products 
in the arena of user interface management systems like 
Interviews (Interviews. 1987; Linton. 1989) or 
CommonView (Glockenspiel. 1989). 

The hierarchy has some properties found in object- 
oriented systems: new objects can be derived from 
existing objects; objects inherit behavior and data from 
their ancestors and re-use code from them. This 
technique simplifies the creation of new interactors 
which on the one hand can have more powerful features 
than their ancestors and on the other hand can be 
restricted in their behavior. Deriving new objects from 
existing ones can be done without programming but 
with the help of modem graphical user interface 
techniques. A set of useful functions (including 
mathematical ones) must be made available for the 
author. Nevertheless it is important to give experts a 
chance to include programmed sequences (e.g. written in 



C or PASCAL) which increase the high flexibility of 
object oriented hierarchy of interactors. 

An example of the advantages we get by deriving new 
objects from existing ones is a DragButton which is an 
area on the screen which can be clicked on and - as far as 
it is connected to graphics - can be moved together with 
its graphical representation to another place on the 
screen. We can make this type of interactor much more 
powerful if we derive a new interactor from it called 
DistDragBuiion which has the same features as the 
DragButton. i.e. clicking on the DragButton. moving it 
around and placing it to a special location, plus a new 
feature which calculates the distance between the actual 
position of the DragButton (i.e. the jyosilion of the 
mouse cursor) and a given path which - for simplifying 
the calculation - must be a polyline connecting the 
starting position and the destiaation position of the 
DragButton. This type of intenictor can be used to 
watch the learner who has to follow a given path. If the 
calcuhted distance exceeds a given value the interactor 
will react in a way that can be :jpecified by the author 
(e.g. with a beep) . 
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Figure 1. 

A standard library contains the most important and most 
frequently used interactors. The more we concentrate 
different types of interactors in the standard library the 
more powerful the system will be. Figure 1 presents a 
collection of different interactors which we think should 
be a part of the standard library. 



The basic idea of how an interactor reacts to user input 
is the capability to receive and send messages. Messages 
are in ge.ieral a main characteristic of object-oriented 
programming languages. They are responsible for the 
communication between objects. HyperCard 
(HyperCard. 1988) is a good example of a system that 
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extensively uses messages in connection with its own 
object-oriented hierarchy. 

Each inteiactor is in one of two states: awake or asleep. 
Both awake and asleep are messages which can be sent 
and received by any interactor. When an interactor 
sleeps, it does not react to any messages (and hence, 
neither to user input) except the one message that sets it 
awake. Only after the interactor has been awaked, it will 
react according to its type. How it is awaked (and put to 
sleep again) is explained below. 

In addition to the messages awake and asleep each 
interactor has its own typical messages it can serve. A 
PushButton for example can receive the message click 
which means that the mouse button was pressed with 
the cursor positioned inside the button area. The 
PushButton then automatically starts a set of actions 
which is associated with the specific message click. 



This action sequence varies with the individual and must 
be linked to the message by the author (as shown in 
Figure 2). Some messages are common to several 
interactors, since the capability to receive messages can 
be transmitted to the descendants of the interactor. The 
basic interactor Tlnteractor, for instance, which is at the 
root of the interactor hierarchy can serve four n cssages, 
namely the two already known messages awake and 
asleep, and new and close. All four must be available 
for all of the other interactors as well. The message new 
is sent when an interactor appears for the first time in a 
lesson, whereas close is used when the interactor 
disappears (afterwards it cannot be used anymoie). Any 
action sequence can be linked to tiiese messages. For 
example, new can be used to draw graphics onto the 
screen which belongs to the interactor and close can be 
used to erase all "rubbish" before an interactor itself is 
deleied 
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In the following we want to give a short description of 
all the interactors listed in Figure 1, their characteristics 
and application: 

TV iteractor: is the basic interactor. It is an ancestor of 
all other interactors and has fundamental features 
which are common to all other interactors (serves 
the messages new and close, awake and asleep). 

Button: A Button is an area on ilie screen (rectangular 
area by default) that can react to mouse input by 
starting an associated set of actions. It depends on 
the Button type which sort of input can be served. 

TouchButton: A TouchButton is an area on the screen 
that reacts to cursor movement. Three different 
types can be distinguished: 

touch Jn starts a set of actions v hen the mouse cursor 
is moved from the outside to the inside of the 
TouchButton. 



touchyut starts a set of actions when the mouse cursor 
is moved from the inside to the outside of the 
TouchButton. 

stilljn repeatedly starts a set of actions as long as the 
mouse cursor stays inside the TouchButton. 

PushButton: A PushButton is a Button that can be 
clicked on. This is the standard type of interactor 
and will be the most frequently used interactor as 
well. 

DragButton: A DragButton has like all buttons an area 
on the screen associated with it. By clicking into 
this area and keeping the mouse button pressed, the 
DragButton can be moved around the screen. Some 
graphic elements can be associated with a 
DragButton, and they will move correspondingly. 
There are two sets of actions associated with a 
DragButton: the first is executed if a DragButton is 
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moved to the correct destination, the second one is 
executed otherwise. The default for moving a 
DragButton to a wrong location is no ^-caction at 
alL 

RadioButton: A RadioButton consists of several 
PushB»:uon-like interactors. Only one of them can 
be set at any time. When the user clicks on a 
RadioButton, it will be set, and the previously set 
button will be reset. The author of a lesson can 
specify a set of actions for each RadioButton. The 
active set for the pressed RadioButton will be 
executed as soon as the student clicks on it. A 
typical use of RadioButtons is a Multiple Choice 
Question with only one valid answer among a 
number of given choices. 

CheckButton: CheckButtons are similar to 
RadioButtons, but several of them can be set 
simulta^ieously. Each CheckButton has its own set 
of actions. CheckButtons can be used for Multiple 
Choice Questions with more than one correct 
answer. 

Slider: A Slider is ^ small icon which can be set to 
different states like a RadioDial or a ScroHBar, 
Sliders mostly have a more complex structure than 
other interactors. A slider may consist of several 
interactors of other types (see the Scrollbar in 
Figure 3 for instance). Sliders come in two 
variations: discrete and continuous. A discrete 
Slider has actions associated with each of its 
possible states, whereas a continuous Slider returns 
a nunieric value between author-deftned boundaries. 

ScrollBar: A ScrollBar is mainly used for scrolling in 
text documents but can also be used for adjusting 
any analog value which does not have to be set to 
an exact number. 



RadioDial: A RadioDial is similar to a ScrollBar in its 
functions. 

Text: An interactor of type Text allows the user to enter 
simple texts. This is the basic interactor for a 
number of derived interactors which interpret the 
entered text in a certain way (e.g. analysis of free 
text answers which is not covered in this pap>cr). 

GeneralText: The GeneralText interactor allows the 
student to enter a text string. 

RealText: The RealText interactor allows the student to 
enter a real number. The number is checked against 
validity. 

IntegerText: is used for integer numbers, IntegerText is 
similar to RealText. 

Timer: A Timer is used for clock functions. A new 
message to the Timer interactor starts an internal 
stop watch. This value can be used for time-outs, 
i.e. after a certain amount of time a set of actions is 
executed which for example gives the student some 
hints about the problem or ends the problem 
because the time assigned to it has run out. 
Another useful application for the Timer interactor 
is the possibility to send messages which are to be 
posted periodically. 

Record: When the Record interactor is opened (new 
message) recording starts and all messages are 
stored in a list This list can be played back at any 
later time. Previously stored playback lists can be 
accessed and can show a confused student the 
solution to a problem by showing step by step the 
correct way to the end of the problem. Recording 
can be interrupted by setting the interactor to 
asleep. 



Scrollbar: 



DragButton 



PushButtons 
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Figure 3. 
The itemilst 

Description of the Itemlist. Two different steps 
are necessary to make an interactor available in a lesson: 
the flrst stq) is the definition of the interactor, the 
second one its activation. To activate an interactor 
means to allow the student to use the interactor 
according to its type (e.g. to click on it if it is a 
PushButton). It is the same as setting the state of the 
interactor to awake. When creating a frame (a frame is a 
part of a lesson) it is important that the autlior keeps 
track of the interactors and their order. The itemlist 
gives an overview of all objects of a frame. Besides 



mainly graphical and textual objects the itemlist 
contains information about the interactors that are used 
in a frame, i.e. which interactors are deflned and when 
they can be activated. All objects belonging to a frame 
are shown as a linked list of small rectangular boxes. 
Each box indicates the corresponding entity. The left 
part of Figure 4 shows a short section of an itemlist. 

When a frame is started the execution begins with the 
first object on the itemlist and continues until all 
objects have been executed. In case of a graphic object 
or text the item is displayed onscreen. If the item found 
in the list is an interactor, its state is set to awake. 
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Execution stops at the interactor and halts until the 
student serves the interactor according to its demands, 
i.e. clicking on a PushButton, moving a DragButton to 
its destination etc. The int^actor is set to asleep and 
execution continues with tlie next item of the itemlist 
Typically, the first occunence of an interactor in the 
itemlist will be its definition. Subsequent occurrences 
just contain the name of the interactor and only se. e to 
activate it. Automatic activating and deactivat' of 
interactors is not as flexible in (fealin^: with probi;;ms. 
Therefore it is necessary to set the state of an interactor 
explicitly to the author's demand. How this can be done 
is described later. 

Before we continue with the description of the itemlist 
we give some more detailed information about the 
definition of an interactor. The detmiuon mcludes the 
following steps: 

• choosing the type of interactor The author can 
select an interactor from the object-oriented 
hierarchy which is shown on the screen. 

• giving a name to »he interactor: Each interactor 
must have a name which is unique witliin a 
lesson. The name is used for an identification 
oftheint^actor. 

• setting attributes depending on the type of the 
interactor The position and the destination of 
an interactor can be edited directly on the screen 
by using some kind of simple graphic editor 
Position and destination need not be 
rectangular areas, also polygons may be used. 
In case of DragButtons graphic objects can be 
assigned to the interactor which causes the 
graphic to follow the dragged interactor 

• supplying the names of action-sequences which 
are to be executed when the interactor is 
manipulated by the student 

An aaion sequence is a named group of different 
primitives, usually graphical and textual objects (see 
Figure 4). To provide a possibility to create 
multimedia-like lessons the author can include sound 
sequences and call external programs (written in C or 
PASCAL for example) that mainly have the purpose to 
drive external devices like CD-ROMs or video-players. 
Finally action sequences may contain messages which 
are to be sent to other interactors. For example, tliis 
technique can be used to set interactors to asleep or to 
awake, or even to simulate a click on a PushButton. If a 
new interactor is defined and immediately set to asleep 
in the action sequence of the new message the interactor 
will not be served and execution will continue with the 
next item. The interactor will be served only the next 
time when the execution stops at the same interactor 
An action sequence can be named and stored separately 
on disk. 

Let us consider the example of the engine that has to be 
taken apart. Here, the author would have to do the 
following things: first, the right type of interactor for 



the first bolt to be removed has to be picked (we assimie 
that the engine has already been drawn on the screen). 
Since students should just click on the bolt, a 
PushButton is the right choice. The author gives a 
name to it, like Bolt J. Then the position on the screen 
has to be defined. Finally the author defines the action 
sequence which describes what happens if Bolt J. is 
clicked on: the drawing of the bolt is erased by placing a 
filled rectangle on it, and some of the lines that got 
partially erased are redrawn. This finishes the definition 
of the interacted*. 

After all interactors have been defined the author can 
rearrange the boxes within the itemlist, can perform 
standard cut/copy/puste oper?tions etc. Interactors can be 
duplicated, parts of the definition of an interactor (e.g. 
the destination area of DragButtons) can he copied and 
pasted to another interactor. Clicking on a box of an 
interactor opens a window with all types of messages 
which the interactor can receive (as shown in Figure 4). 
New action sequences can be linked to the messages or, 
if there are already existing ones, these actions can be 
viewed and changed. By clicking on the box twice the 
definition of the associ^Ued interactor is shown and can 
then be manipulated. 

Sequential and Parallel Activation of 
Interactors. The two examples given at t\e 
beginning of this paper show another important aspect 
of interactors. The interactors are usually activated 
sequentially, can be manipulated, and fall asleep again. 
It is nevertheless possible to have several interactors 
awake at the same time. In the case of the engine all 
bolts would be active at the same time, giving the 
student the chance to remove one after the other in 
arbitrary order A typical itemlist for parallel activation 
of interactors is shown in Figure S: it uses branches 
from a single item to several others. As soon as the 
first four PushButtons have been pressed (which 
corresponds to removing four bolts), the next 
PushButton can be clicked on (which corresponds to 
removing the part that was fixed by the four bolts). 

The given example expects that each of the interactors 
in the parallel structure has to be served, i.e. bolt J and 
bolt_2 aiLl bolt_3 and boltjf. However, the author 
often has to present a problem where several interactors 
are active simultaneously but only one of them (it 
makes no difference which one) has to be served, 
utboltj Qibolt^ Qlboli^j Qlboltjt. Therefore we 
need an additional structure which helps us to solve the 
problem. We want to show now that both parallel 
structures (and, or) are in fact no new structures and that 
both can be solved very easy by using the messages 
awake and asleep. At the beginning all corresponding 
PushButtons have to be awaked. In the case of the and- 
structure each PushButton that has been clicked on is 
automatically deactivated while the other PushButtons 
still have to wait. 
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Figure 4* 

When all PushButtons have been clicked on no 
PushButton remains awake and execution is continued 
with the next item after the parallel structure. In the 
case of the or-structure the PushButton that has been 
clicked on is also set to asleep. Additionally, all other 
PushButtons have to be deactivated with the result that, 
after the Hrst PushButton has been clicked on, no 
PushButton of the or-structure remains awake and 
execution continues with the first item after the or- 
structure. 

Although parallel structures follow simple rules it 
would be too a implicated for the author to set all asleep 
and awake messages. Hence it is useful for the editing 
of such structures to use the graphical representation. 
This simple scheme allows authors to create moderately 
complex interaction patterns without programming. 

Some More Examples 
Piano Keyboard. Think of the following problem: 
Students of music shall be encouraged to train the 
sensitivity of their ear to different notes. A piano 
keyboard is shown on the screen and a short mdody is 
played by the computer. The students have to replay the 
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new 
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sequence exactly by clicking with the mouse on the 
piano keys in the right order. Errors are immediately 
punished with a squeaky tone. 

The first thing to do for the author is to draw the piano 
keyboard on the screen. Afterwards a PushButton has to 
be related to each of the keys. Each PushButton has a 
typical tone (with a specific pitch) which is stored in 
the action sequence. To serve the squeaky tone as well a 
second PushButton has to be defined for each piano key 
with an action sequence that plays the squeaky tone. All 
PushButtons with squeaky tones are set to awake at the 
beginning, all others except the one which is the first to 
be clicked on by the student arc set to asleep. If the 
student clicks on the right piano key the tone is played 
and the interactor is set to asleep afterwards, whereas the 
corresponding PushButton with the squeaky tone is set 
to awake. The whole process - that is playing the tone, 
setting itself to asleep and awaking the other 
PushButton - must be done by one and the same action 
sequence. Setting the squeaky-tone PushButton to 
asleep must be done by the action sequence that belongs 
to the awake message of the PushButton with the 
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correct tone. If the interactors are defined that way it is 
very easy then to create the itcmlist. The itemlist is 
nothing else than a serial list of PushButtons that have 
to be waked up. 

The given example expects that each of the interactors 
in the parallel structure has to be served, i.e. boh J and 
boUJ?, QSiiboU^S and boitj$. However, the author 
often has to present a problem where several interactors 
are active simultaneously but only one of them (it 
makes no difference which one) has to be served, 
ucboltj QiboltJ2 01 bolt J Q£ bolt J. Therefore we 
need an additional structure which helps us to solve the 
problem. We want to show now that both parallel 
structures (and, or) are in fact no new structures and that 
both can be solved very easy by using the messages 
awake and asleep. At the beginning all corresponding 
PushButtons have to be awaked. In the case of the and- 
structure each PushButton that has been clicked on is 
automatically deactivated while the other PushButtons 
still have to wait. When all PushButtons have been 
clicked on no PushButton remains awake and execution 
is continued with the next item after the parallel 
strucuire. In the case of the or-structure the PushButton 
that has been clicked on is also set to asleep. 
Additionally, all other PushButtons have to be 
deactivated with the result that, after the first 
PushButton has been clicked on, no PushButton of the 
or-strucuue remains awake and execution continues with 
the first item after the or-stnicture. 

awake tone c, awake tone d, awake tone a, ... 



Assembling an Engine 

The second example is similar to our example with the 
engine that has to be taken apart, only this time it is 
the other way round: we have a number of single pieces 
lined up on side of the screen which belong to an 
engine. What the student has to do now is to take each 
part and move it to the conect kx:ation on the screen, so 
that at the end we have an engine that would be able to 
woric - with all parts on the right position. 

This example is quite simple to manage. The first step 
would be to define a DragButton for each part which has 
to be moved, then connect it with the graphical 
representation of each par*, so thai a bolt or a cover is 
moved together with the mouse cursor when the 
DragButton is moved over the screen, and finally define 
the destination locations for each part of the engine. 
After a deflnition of the DragButtons we have a 
disordered set of interactors collected in the itcmlist The 
next step would be to order the DragButtons in the 
itemlist by moving them around and using 
cut/copy/paste operations, according tc the demands of 
the problem (some pieces may not be moved before 
other pieces have been placed). In the end the action 
sequences Iiave to be created and linked to the interactors 
and the definition of the problem is finished. 

Conclusion 

We have shown how tutorial lessons can be enhanced 
using the simple, yet powerful concept of interactors. 
Students have to be offered an active role in working 
through lessons: it makes learning more interesting, 
challenging and woLJd result in better subject 
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understanding. Sjiudcnts can click on Buttons, move 
DragButtons around, enter texts etc. Depending on the 
user input action sequences are started. An action 
sequence can be a textual teedback presented to the 
student, graphical objects which are drawn onto the 
screen, or even a short video sequence. To react to user 
input we used an extendable object-oriented hierarchy of 
interactors . New interactors with new features can be 
created which - in connection with modem graphical 
user interface techniques - give the system more power 
and flexibility. To give one example of practicable 
future developments think of the heq)sort example once 
more. Until now the authors had to define each 
interactor by hand, and they had to know exactly how 
many numbers had to be sorted. Additional interactors 
which observe special rules would automatically place 
the PushButtons on the screen and thus m^e the 
heapsort problem independent of the quantity of 
numbers to be sorted. 
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Abstract 

The need for formal, documented l-aining programs is 
growing at an exponential rate, and most of these 
programs must be performance based. As a result, we 
have an extreme need for automated tools that will 
allow us to make the best use of the limited number of 
professionals qualiOed to develop such training 
programs. In this paper, we describe the development of 
one such tool, Questionnaire/Task Analysis (QTA) that 
automates some parts of the job/task analysis process. 
When provided with an appropriately validated task list^ 
QTA can be used on a computer to administer the 
questionnaire that allows the job incumbents to 
evaluate, on a scale from i to S, the frequency of 
occurrence, importance, and difficulty of 
performance for each of the tasks on the list. It can 
then analyze the data collected from one respondent or a 
group of respondents, according to a decision tree 
specifically programmed into QTA, to make 
recommendations about the level of training required for 
each of the tasks. QTA also prepares a report 
summarizing the task listt the average frequency, 
importance, and difficulty ratings for each task and 
the traimiig recommendations for each task. 

Introduction 

As members of A-6, the Cognitive Systems 
Engineering (CSE) Group at the Los Alamos National 
Laboratory (LANL), we recently performed a job/task 
analysis (JTA) for a group of people responsible for 
operating the control room of a large building complex 
used for research on radioactive materials. In this paper, 
we will refer to this group as the control room operators 
(CROs). This JTA was to be used to develop a 
pcrformance-based training (PBT) program for these 
CROs. 

In this paper, we will first discuss in general terms the 
process of developing a PBT program. Next, we will 
look more closely at the requirements of performing a 
JTA, discussing in detail those portions of the JTA 
process that can be performed using our 
Questionnaire/Task Analysis (QTA) program. Finally, 
we will discuss the structure, requirements and 
limitations of QTA. 



Overview oC Performance-Based Training 
Deveiopment Process 

The process of developing a performance-based training 
program (sec, for example Gagnd, Briggs, & Wager, 
1988) involves numerous steps, many of which must 
be conducted in sequence because they form the basis for 



the next step. The major steps of the PBT development 
process are listed below. 

1. Perform a Needs Analysis - Determine if 
training is needed to solve existing problems. 

2. Perform a Job Analysis - Develop a valid duty 
area/task list. 

a. Determine what specific duty areas make 
up the job and what tasks are contained in 
each duty area; 

b. Produce a validated list of those duty areas 
and tasks; 

c. Evaluate each task to determine its 
difficulty, importance, and frequency of 
performance; and 

d. Analyze the above task evaluation data to 
determine which tasks require training and 
at what level. 

3. Validate the Task Training Requirements. 
Using a committee made up of subjert matter 
experts (SMEs), supervisory personnel, and 
job incumbents, review the task training 
recommendations from Step 2(d) to verify the 
level of training recommended for each task. 

4. Perform a Task Analysis. For those tasks 
requiring training, determine the detailed 
bubtasks (i.e., task elements) required to 
accomplish the task in the proper order, 
including all conditional branching. 

5. Revic Existing Training Materials 

6. Prepare a Tas^ -lo-Tnuning Matrix 

7. Establish Terminal and Enabling Objectives 
for Each Task and Element 

8. Generate the Training Materials 

9. Evaluate the Training Matcnals 

10. Revise the Training Materials 

1 1 . Repeat from Step 1 as Required 

A More Detailed Look at the 
Job/Task Analysis Process 

A job/task analysis (Carlisle, 1986) is generally 
considered to include all of the activities in Steps 2 
through 4 above; Steps 5 and 6 can also be included, 
depending on the status of the training program at the 
lime the JTA is conducted. In the case of a job where no 
training program exists at the time of the JTA, Step 5 
is irrelevant and Step 6 is better performed after the 
training materials are generated. 

In this paper we will concentrate on Step 2 above 
because that is the part of the JTA process that is ad- 
dressed by our QTA program. 

One of the requirements for our JTA was that it had to 
be conducted according to a Training Accreditation Task 
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Order recently implemented by the U.S. Department of 
Energy (DOE) (DOE 5480.18. 1989). This Task Order 
recommends that the task evaluation process (Sicp 2[c] 
above) be conducted by means of a questionnaire. This 
questionnaire lists the individual tasks and asks the job 
incumbents to rate the frequency, importance, and 
difficulty of each task on a scale from ! to S. In 
addition, a zero (0) response is allowed on the frequency 
rating to indicate that the incumbent never does the task 
(Definitions of these scales are listed in the ^pendix). 
The questionnaire is to be administered to a 
representative sample of job incumbents and 
supervisors. The frequency, importance, and difficulty 
rankings of each respondent for each task are then 
averaged and analyzed by means of a decision tree to 
determine the training level required for each task. Three 
training levels are recognized: no train, train, and 
overtrain. These terms are defined as follows: 

No Train - The task can be learned in the normal 
course of job activities and, therefore, requires no formal 
training. 

Train - The task requires some type of formal training, 
which Ga. include on-the-job training (OJT), classroom 
lectures, computer-based training (CBT), simulation- 
based training (SBT), etc. 

Overtrain - The task is difficult and/or important 
enough to require training; and, in addition, it is 
performed so infrequently that the incumbents cannot 
maintain competence via actual task performance. 
Therefore, periodic retraining is required 

The decision tree from DOE 5480.18 is shown in Fig, 
1, with the break points for each of the three factors 
(frequency, importance, and difficulty) indicated in the 
figure. In our experience, the training recommendations 
resulting from this decision tree analysis are not very 
intuitive. As a result, we summarize the general 
frequency, importance, and difficulty characteristic of 
tasks recommended for each of three possible training 
levels in Table I. With this baci^g^)und in the JTA 
process, we start our detailed examination of QTA in 
the next section. 



Purposes of QTA 

QTA is a computer {M'ogram that was developed to 
perform three primary functions and is appropriately 
used for a fourth function in some cir.-'mstances. Its 
three primary functions are as follows: 

1. Electronically (i.e., using a computer) 
administer the task evaluation questionnaire 
and collect and store the data (i.e.. Step 2[c] of 
the PBT process). 



2. Analyze the collected data according to the 
decision rxcc in Fig. 1. 

3. Prepare a Task Training Requirements report 
according to the recommendations of DOE 
Order 5480.18 listing the tasks, the average 
frequency, importance, and difficult ratings, as 
well as the training level recommendations, for 
each task (i.e.. Step 2[d] of the PBT process). 

As a fourth function, the [Hogram can also be used as a 
data entry and analysis tool in those cases where a 
printed paper version of the questionnaire must be filled 
out by hand. We used this method several times and 
found that entry and validation of the data into QTA is 
quite time consuming. In addition, our experience to 
date suggests that even computer-phobic people can 
effectively use QTA to complete the questionnaire. As a 
result, we recommend computer administration unless 
the requisite computer hardware is not available. 



Structure of QTA 

QTA is written in Modula-2 (LOGITECH, 1987) and 
runs under DOS (Version 2.0 or above) on any IBM 
PC, XT, or AT compatible computer that has a color 
graphics output capability. For QTA to run property, 
the DOS ANSI.SYS driver must also be loaded QTA 
can run from any floppy disk or from any 
directory/subdirectory on a hard disk, provided that the 
data files it requires (see below) are present in the same 
location. It will write its output Hies to this same 
location. The executable, compiled version of the 
program (QTA£XE) occupies about 90 kbytes of RAM 
and requires about 150 kbytes of RAM to execute 
prq)erly. 

The QTA program provides file handling and 
generation, questionnaire administration, data gathering, 
data analysis, and report generation functions. To 
operate property, it requires two additional ASCII data 
files: TASKS.DATand EVAL.DAT. The TASKS.DAT 
file consists of a JTA title followed by a list of the duty 
areas with their corresponding tasks. To clarify this 
fliscussion, we present the task list for a hypothetical 
job in Tabic II and the TASKS.DAT derived from it in 
Table III. As QTA is presently implemented, there are 
three consu^ints on the TASKS.DAT file: 

1. The duty area description must be on a single 
line. 

2. The task description must be on two lines. If 
there is not enough text to occupy a second 
line, some character other than a line feed, such 
as a period, must be entered into the second 
line. 

3. No line of text may contain more than 70 
characters (including any item number). 
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Difficulty 



Importance 



Frgqufflcy 



Recommendation 



VEBY DIFFICULT 
i.35 



IMPORTANT 
^23 



UNIMPORTANT 
<2S 



MODERATELY 
DIFFICULT 

>''.54<35 



IMPORTANT ^ 
i25 



UNIMPORTANT 
<2S 



EASY 
525 



IMPORTANT 
i25 



UNIMPORTANT 
<2S 




NO TRAIN 
NO TRAIN 
TRAIN 
NO TRAIN 

NO TRAIN 
NO TRAIN 



Figure 1. Decision tree used in analysis of questionnaire data. 
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Qifficulty Importance Fregugncv 

Vciy Voy Often 

>3.5 ^2.5 >3.5 

Modeiaie Occasionally 

> 2.5 & < 3.5 > 2.5 & < 3.5 

Easy Not Seldom 

^ 2.5 < 2.5 < 2.5 



The EVAL.DAT file is an ASCII file thai describes the 
structure of the TASKS.DAT file as a linear array of 
integers. The first integer is the number of duty areas in 
the TASKS.DAT file, followed in order by the number 
of tasks in each of the duty areas. A duty area may have 
zero tasks, which allows the inclusion of subheadings 
within the rqwrt finally generated by QTA (see below). 
The EVAL.DAT file for our hypothetical task list is 
presented in Table IV. 



T able 2. Task List for Hvt)0thetical Job 

102. PROCESS COOLANT SYSTEM 

102.1 Prepare tank car for unloading. 

102.2 Unload tank car. 



111. AREA CONTROL ROOM 

111.1 Monitor assay ^)ectrometer 

111.2 Calculate assay 
1113 Adjust line recorders 



Table 3. TASK.DAT File for Hypothetical Job 



102. reOCESS COOLANT SYSTEM 

102.1 Prepare tank car for untoading 

102.2 Unload tank car. 

1 1 1. AREA CONTROL ROOM 

111.1 Monitor assay spectrometer.to 
determine background 

11 1.2 Calculate assay. 

1 1 1.3 Adjust line recorders. 



Table 4. EVAL.DAT File for Hypothetical Job 



Job Analysis for Hypothetical Job 

1 

2 
3 



When the questionnaire is administered by QTA, it 
builds two data files. The first of these is an ASCII 
listing of the respondent's frequency, importance, and 
difficulty ratings for each task. In our present version of 
QTA, this file is named employee number.cat 
where "employee number** is the employer-assigned 
number of the respondent and "cat** is a three character 
extension representing the employee category, either 
operator (OPR) or supervisor (SUP) in the present 
version of QTA. The second file built by QTA is 
FILELIST.DAT, which is simply a list of the 
respondent questionnaire data files generated by the 
pr'tram. 

Operation of QTA - Questionnaire 

When QTA is first started, it requires the respondent to 
choose from four possible functions to be performed, 
namely the following: 

1 . Complete the "long form" of the questionnaire. 

2. Complete the "short form" of the 
questionnaire. 

3. Analyze the results. 

4. Exit the program. 



Table L Summary of Decision Tree Criteria a 
Rating Scales 

NO TRAINING IS REQUIRED FOR 
THE FOLLOWING:* 

• Easy tasks unless they are both important and 
infirequently performed 

• Moderately difficult tasks if they ars not 
inqKXtant and are performed nKxIerately to 
very often 

TRAINING IS REQUIRED FOR THE 
FOLLOWING: 

• Easy tasks that are important and are 
perfonned infiequently 

• Moderately difficult tasks that are (1) 
impoflant and perfonned very often or (2) not 
important and are performed infiequently 

• Very difficult tasks that sre impodan t 
provided they aie perfonned very ofken 

• Very difficult tasks that are not impcNtant if 
they aic perfonned nx)derately often or only 
sddom 

OVERTRAINING IS REQUIRED FOR 
THE FOLLOWING: 

• Both very difBcuIt and moderately difficult 
tasks that are important and are perfonned 
only moderately often or seldom 



*Note: The break points for our rating scales of 1 to 
were set as follows: 
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If cither option 1 or 2 is chosen, the program then 
collects demographic data from the respondents, in- 
cluding their name, employee number, job title, 
section, employee category, years of experience, and 
education. All of this information is entered into this 
respondent's data file. Then QTA administers the actual 
questionnaire by presenting the tasks one by one, in 
order. For each lask, the duty area is listed at the top of 
the screen followed by the description of the task, both 
obtained from the TASKS.DAT file. 

If the "long form" was chosen, three individual, 
consecutive screens are generated for each task to obtain 
the level ranking for each of the evaluation factors (i.e., 
frequency, importance, and difficulty) for the displayed 
task. This format allows enough room on the computer 
display screen to provide on-screen definitions of the 
rating scales. If the respondent enters a zero for 
frequency on the first screen, the importance and 
difficulty screens for that task are skipped and QTA goes 
on to the next task. The advantage of the long form is 
that the definitions of the evaluation scales are on the 
screen with the task description, eliminating the need 
for a printed reference. The disadvantage is that the 
respondent must view more screens than if he had 
chosen the short form (sec below). 

If the respondents choose to complete the short form of 
the questionnaire, they are first presented with a screei\ 
that asks whether they ever do the task. If they respond 
with a yes, they are presented with a single screen t!iat 
requests the level ranking for all three evaluation 
factors. Because of space limitations on that screen, the 
short form docs not provide rating scale definitions on 
the screen. As a result, when using the short form, we 
must provide the respondents with printed listings of 
the scale definitions for all three evaluation factors. 

Actually, when QTA is used to administer the 
questionnaire on the computer, we recommend that the 
respondents be provided with printed copies of both the 
task list and the definitions of the rating scales, re- 
gardless of whether the long or short form is chosen. 
The printed task list enables the respondents to ascertain 
the relationships among tasks and duty areas. The 
printed rating scales are used as a reference to help 
clarify the rating choices. 

As described above, the data collected by QTA are stored 
in convenient ASCII files: as a result, they can be 
analyzed by tools other than QTA, such as spread 
sheets, data base management programs, etc. 

Operation of QTA • Data Analysis 

As described above, QTA creates an individual data file 
for each person completing the questionnaire; these 
questionnaire data files are used in the analysis phase to 
be discussed now. QTA also keeps a list of these 
individual data files it has created in the FILELIST.DAT 



file. This latter file is also accessed during the analysis 
portion of the program to determine which questionnaire 
data files to include in the analysis. 

As presently implemented, QTA allows two job types 
to be entered during the demogr^hic data collection of 
the questionnaire; these are operator or supervisor. As a 
result, the questionnaire data files it constructs are of 
two types, namely *.OPR or *.SUP.. When QTA starts, 
if the analyze fuixtion is chosen from the opening 
screen, the analyst is given the choice of analyzing the 
data from the operators, the data from the supervisors, 
or the combination of both sets of data to determine the 
training recommendations resulting from the decision 
tree analysis. The data from the appropriate files are 
then used to compute the numerical averages for each of 
the rating factors of each task. These average rating 
factors are then analyze-d using the decision tree to 
obtain the corresponding training recommendations. 
After the analysis is completed, QTA creates an analysis 
file. This file includes a header with the name of the 
survey; the category of questionnaire data files analyzed; 
and for each task, the task name» the averages of the 
frequency, difficulty, and importance ratings, and the 
recommendation for the training level for that task. An 
output file for our hypothetical job is presented in Table 
V, literally the computer ou^uL 

The analysis files for the separate data sets Ci.e., 
supervisors, operators and all respondents) should be 
compared to establish the validity of the 
recommendations. Because QTA uses an ASCII file, 
FILELIST.DAT, to define which questionnaire data files 
are averaged for any single analysis, the analyst may 
edit that FILELIST.DAT file to include or exclude 
individual questionnaire data files from any particular 
analysis. Thus, if any single individual's responses 
differ greatly from the responses of his peers, the effect 
of that individual's data on the analysis of the averaged 
results can be explored by running the analysis first 
including and then excluding that data file and 
comparing the results. 

Again, as mentioned above, the analyst can easily 
access the individual files with a word processor or 
ASCII editor and then use the files with other analysis 
tools, such as spread sheets, data base management 
systems, etc.. 

Limitations of qta 

QTA has several limitations. One limitation of the 
current version of QTA is that the decision tree logic 
used in the data analysis is "hara wired" into the 
program at the Modula-2 code level. If someone wants 
to use the program with different break points or with a 
different logic in the decision tree, it is necessary to 
make those changes in the original Modula-2 source 
code. 
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Table 5, Task Analysis Report Output From QTA for Hypothetical Job 



JOB ANALYSIS TABULATION 

Job Analysis for Hypothetical Job; all personnel 

CompOcd By: 

Verified By: D«c: 

RaledBy : Vkiti 



TASKS 
Comments 

102. PROCESS COOLANT SYSTEM 

102.1 Prq)are tank car for 1.67 
unkxding 



DIFF 



102.2 Unload tank car 

11 L AREA CONTROL ROOM 

11 LI Monitor assay spectroineter 
to determine backgHDund. 

lllJZCakulate assay 

1 1 1 .3 Adjust line recorders 



2.33 

2.33 

2.67 
1.33 



4.00 
4.00 

4.67 

5.00 
3.67 



IMP 

3.00 
3.33 

3.67 

4.00 
2.67 



FREQ 

No Train 
No Train 

No Train 

Train 
No Train 



Number of Surveys 3 



Another limitation is that there are minimal undo and 
back-up capabilities. When respondents enter their 
fiequency, importance, and/or difficulty ratings, they c^z 
use the back^Mce key or the space bar (dq)ending on the 
situation) to change their responses. However, once 
they depress the ENTER key, they cannot go back and 
change their reqxmses. 

Finally, when respondents are completing the 
questionnaire, they must finish the entire task list 
'Fhere is no mechanism for stopping in the middle and 
then resuming at that place later. As time permits, we 
plan to rectify some or all of these shortcomings. 

What QTA Does Not Do 

QTA is an automated tool f hat was developed to quickly 
and efficiently perform several of the steps requiied in a 
complete JTA. It administers the task evaluation 
questionnaire, analyzes the questionnaire data according 
to one specific paradigm and criteria set, and prepares a 
formatted report summarizing its training 
recommendations. However, this is all that it does. // 
does not do the following: 
^ Generate the task list. 



♦ Validate the task list, 

^ Examine/compare the questionnaire data sets 
for reasonableness, tfioughtlessly entered 
ratings, cooperative responses, overinflation of 
job importance, etc., 

♦ Validate the training recommendation list, 
« Perform the detailed task analysis, 

♦ Etc. 

Anyone planning to use QTA should always keep in 
mind its limitations. It is, after all, just a "dumb" 
computer program, subject to the usual GIGO 
syndrome. If it is fed garbage, either in the form of 
assumptions or as data, its product will most assuredly 
also be garbage! 

Conclusions 

We are now in the process of doing our third job 
analysis using QTA. Although it does have some 
limitations, we have found it to be an excellent tool for 
administering task evaluation questionnaires and ana- 
lyzing the resulting data. Most of the problems we have 
encountered were related to the difficulty of clearly 
communicating to the respondents the meaning of the 
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rating scale values, especially for the importance factor. 
We plan to summarize our experiences in this area in 
another p^r to be prepared soon. 
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Appendix - Definitions of the Task Rating 
Scales 

0. NEVER do this task 
I Less than once a year 

2. Once every 5 to 12 months 

3 . Once every 3 weeks to 4 months 

4. Once every 1 or 2 weeks 

5 . More frequently than once a week 

The importance scale is rated in terms of the 
consequences of improper performance in the following 
impact areas: 

Impact to the environment 

• Consequences to your health and that of others on 
the job 

Security infractions 

• Violation of LANL or ANSI standard procedures 
and/or DOE or DOT regulations 

Quality and quantity of job accomplishments 



The importance scale ratings are listed below: 

1. MINIMAL - Violates standard procedures and/or 
regulations, but there are no consequences in other 
impact areas. 

For example: Improper task performance results in a security 
regulation violation, but the task is accomplished this time 
with no breach of seonity. 

2. UNDESIRABLE - Violates standards and/or 
regulations resulting in minimal consequences 
to some or all of the impact areas. 

For example: Improper task performance may result in a dose 
considered inconsistent with the "as-low-as-reasonably- 
achievable" (ALARA) philosophy, or there is potential 
environmental impact. 

3. SERIOUS - Results in significant consequences to 
some or all impact areas. 

For example: Improper task performance may result in 
{MTsonal injury, but the injured person can return to work, or 
there is material and/or equipment damage requiring significant 
corrective action. 

4. SEVERE - Results in severe consequences to some 
or all impact areas. 

For example: Personal injury occurs that requires 
hospitalization, the environment is contaminated, or material 
and/or equipment damage occurs requiring costly and extensive 
corrective action. 

5. EXTREMELY SEVERE - Results in the most 
severe consequences in some or all areas. 

For example: A criticality accident occurs; there are fatal 
injuries to personnel; or the facility is shut down because of 
equipment, facility, or environmental damage. 

The difficulty scale ratings are listed below: 

1 . Very easy to perform 

2. Somewhat easy to perform 

3. Moderately difficult to perform 

4. Very difficult to perform 

5. Extremely difliculf to perform 
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Abstract 

This paper describes teaching materials designed for 
courses in which a portion, but not all of the time, is 
devoted to expert systems. Knowledge representation, 
backward chaining, confidence factors, multi-value 
variables, knowledge engineering, induction, and shell 
evaluation are covered. A 56-file disk with class 
handouts, assignments, solutions, and knowledge bases 
for assignments and demonstrations is available from 
the author. 

Artificial intelligence pioneer Allen Newell has stated 
that "there is no doubt, as far as I am concerned, that the 
development of expert systems is the major advance in 
the artificial intelligence field during the last decade 
(Newell, 1987)." Newell was speaking from a scientific 
standpoint, but he also acknowledges that expert 
systems "has been the main driver of the current wave 
of commercialization of AI." If Newell's assessment is 
correct, expert systems should be included in the 
cuniculum. 

This paper begins by tracing the evolution of expert 
systems from research to the personal computer. Next 
we discuss the place of expert systems in the 
curriculum. Finally, we describe machine-readable 
teaching materials developed for introducing expert 
systems. 

Background 

Expert systems trace their roots to the foundations of 
formal logic, and early attempts to autonoate deduction. 
When computers became available, pioneers like 
McCarthy (1960, 1988) claimed that we should 
concentrate our efforts on representing knowledge, 
letting an automatic reasoning program draw relevant 
conclusions. At the same time, Herbert Simon and Alan 
Newell began trying to capture expert's strategies in 
programs. They recorded "protocols" of graduate 
students thinking aloud while proving theorems, then 
wrote programs called the Logic Theorist (Newell, 
1963) and General Problem Solver (Newell, 1963) 
which applied the same sort of pattern-matching and 
difference-reducing heuristics as the students. 

The forerunners of modem expert system shells 
(devetopmen: environments) were developed at Stanford 
University, First came DENDRAL (Lindsay, 1980), a 
program for enumerating the set of feasible molecular 
suiictures of a chemical compound given spectrograph 
input, and trying to decide which suiicture was most 
likely correct. Several systems were inspired by 
DENDRAL, including MYCIN (ShorUiffe, 1976), a 



program to aid in the diagnosis and treaunent of 
meningitis and bacteremia infections, given input from 
a physician. The fu-st shell was E.MYCIN (Buchanan. 
1984) (Essential MYCIN) which separated the MYCIN 
inference program from the knowledge base of 
diagnostic heuristics. EMYCIN was MYCIN with the 
domain-specific knowledge removed, so some call it 
"Empty MYCIN." 

These early expert systems and shells required large 
computers and projects, but that is no longer the case. 
Today, there are several shells which are practical for 
development and delivery on standard, 640 KB, MS- 
DOS computers (Press, 1988). One of these, Texas 
Instrument's PC Plus is based on EMYCIN. 

The feasibility of low cost expert systems is illustrated 
by the experience of Ed Mahler, Director of the 
Artificial Intelligence Division at Du Pont (Mahler, 
1990). Du Pont has deployed over 200 expert systems. 
Each is quite small (averaging SO production rules), but 
Mahler estimates the aggregate savings to Du Pont as 
$40 million annually. It is noteworthy that these 
systems are not implemented by staff programmers, but 
by end users who take a two-day u^ning class and are 
supported by Mahler's depaitmenL 

Expert Systems In the Curriculum 

Since expert system technology is becoming widely 
available, a full-semester, elective course in expert 
systems is justified for computer science (CS) and 
computer information system (CIS) majors. Bahill and 
Ferrell (Bahill, 1986) at the University of Arizona have 
taught and written about such a course. In addition to 
teaching useful skills, the course is a good introduction 
to declarative programming. Expert system shells allow 
students to rapidly develop meaningful programs, so the 
declarative programming "forest" will be clear without 
examining too many "trees." 

Expert systems may also be taught as a portion of an 
Introduction to Artificial Intelligence (see, for example, 
the ACM model curriculum course CS 12 (Austing, 
1979)). Most texts for the Introduction to Artificial 
Intelligence course include a discuriion of expert 
systems, for example (Chamiak, 1985). This course 
also typically introduces list and declarative 
programming concepts with Lisp or Prolog exa-cises. 
The relatively low level of these languages and the 
amount of material which must be covered in the course 
restrict the programming exercises to simple examples. 
More comprehensive assignments are feasible in a 
limited dme when using an expert system shell (Koster 
1988). 
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Expert systems are also relevant in an Introduction to 
Decision Support Systems (sec, for example, the 
DPMA model curriculum course ClS/86-11 (DPMA, 
1986)). as evidenced by texts such as Dohcrty and Leigh 
(Leigh. 1986). This course typically focuses on 
techniques and tools, with no programming. Here again, 
expert system shells enable us to introduce 
programming exercises in the limited time available. 

A module on expert systems might also be appropriate 
in a cognitive psychology course or in course in a field 
in which there are substantive applications of expert 
systems. For example, an expert systems module is pan 
of a pre-law course, offered in the sociology department 
at our university. 

With these needs in mind, we developed material for an 
expert system module which can used as part of a course 
or as the beginning of a course dedicated to expert 
systems. The material has been tested in a Decision 
Support Systems course for MBA students. In addition 
to expert systems, that course covered optimization 
techniques, simulation, spreadsheet modeling, project 
management, statistical forecasting, executive support 
systems, and group decision support techniques. 

Machine Readable Teaching Materials 

All of the material described below is available in 
machine-readable form, so that it may be easily copied 
and edited. The distribution disk contains handouts, 
homework assignments, knowledge bases for 
assignments and demonstrations, and solutions to 
assignments. 

Programming assignments and examples use the VP 
Expert (Paperback Software) shell. VP Expert is 
compared to several other shells in Press (Press, 1988), 
and found to be suitable for instructional assignments. 
Sondak (Sondak, 1986), reaches the same conclusion. 
Other commercially available shells would have been as 
good, but none were as reasonably priced. 

The topics covered are knowledge representation and 
backward chaining, confidence factors, multiple values 
for variables, knowledge engineering, induction, and 
shell evaluation. Let us begin with knowledge 
representation and backward chaining. 

Knowledge Representation and Backward 
Chaining. The first set of demonstration knowledge 
bases, handouts and assignments introduce the basics of 
knowledge representation and automatic inference using 
backward chaining. Six simple knowledge bases, 
FOOD1-FOOD6, are used to illustrate these basic 
concepts as well as some of the VP Expert user 
interface options. They are used for in-class 
demonstration and discussion and for assignments. 

Figure 1 is a listin^^ of FOODl.KBS and Figure 2 
traces its execution. FOODl illustrates the procedural 



action section, rules, and declarative statements of VP 
Expert knowledge bases. The four rules are easily traced 
in class and in a handout, providing an illustration of 
backward chaining. The file FOODTRC.HDT is a 
handout explaining the execution trace of FOODl. 

F00D2 through F00D6 are variations on FOODl, 
illustrating the VP Expert keywords and statements: 
because, display, autoquery, runtime, ask/choices, and 
endoff. The following shows which constructs are used 
in each: 

FOODl: runtime, ask/choices, display 
FOOD2: runtime, ask/choices, e^.doff, display 
F00D3: runtime, ask/choices, display, autoquery 
F00D4: runtime, ask/choices, because, display 
FOODS: ask/choices, display 
F00D6: autoquery, runtime, display 

The ideas of declarative programming and backward 
chaining are new to most students, so the di'>tribution 
disk contains two assignments requiring them to trace 
knowledge base execution and to work with the VP 
Expert editor and user interface. 

In FOOD1-FOOD6, all variables are boolean or 
character strings, much like those in a procedure- 
oriented language. However, variables in a shell differ 
from those in procedure-oriented languages in that they 
have varying degrees of certainty and they may assume 
more than one value. The materials in the following 
two sections illustrate each of those differences. 

Confidence Factors. Confidence factors arc 
illustrated by a handout, an assignment, and seven 
knowledge bases for use in class demonstrations. The 
first demonstration knowledge base CNF.KBS shows 
how to input confidence factors and shows the value of 
the toiih threshold for confidence factors. The next two, 
CNFOR and CNFAND, demonstrate the affect of 
disjunctive and conjunctive conditions in the i/ portion 
of a rule. CNFOR is documented in CNF.HDT. a 
handout which can be used along with the 
demonstration. 

Three other knowledge bases, CNFPROB, CNFPROR, 
and CNFPRAlsfD, are used in class demonstrations to 
illustrate the affect of confidence factors in the then 
portion of rules. CNFPROB is simplest, using a single 
variable rule, so it is demonstrated first. CNFPROR and 
CNFPRAND add disjunction and conjunction to the if 
portion of the sample rule. 

CNF.AST is a three part assignment on confidence 
factors. The student first experiments with conjunction 
and disjunction in the if portion of rules, then with 
confidence factors in the then portion of rules. 

Multiple Values for Variables. Shell variables 
also differ from those in procedure-oriented languages in 
that they can assume many values. There are two 
knowledge bases and a hanaout illustrating the idea of 
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plural-value variables. They are variations on FOODl. 
In FOODPLl, all three variables are plural; in 
FO0DPL2 only FOOD and WINE^COLOR are plural. 
These aie explained in a handout file. 

Knowledge Engineering* Once the students are 
familiar with the shell, we turn to a knowledge 
engineering. In addition to theoretical discussion, the 
students do thiee thiee rapid prototyping assignments: 

KEl: The first of three, graduated knc ledge 
engineering exercises in which the student builds a 
quick prototype of an expert system for selecting 
DBMS. 

KE2: The second knowledge engineering exercise in 
which the student revises and elaborates upon the 
prototype built in the first 

KE3: The third knowledge engineering exercise in 
which the student adds database access to the knowledge 
base built in the first two. A small DBF file 
accompanies this assignment 

The assignments are woided as **interviews" between the 
domain expert and knowledge engineer, and the student 
is asked to implement a prototype of the system. Figure 
3 shows the interview from the first knowledge 
engineering assignment The full assignment file 
contains some sample runs and further instructions to 
the student 

The second knowledge engineering assignment 
elaborates upon the first and students having d'fficulty 
adapting their thinking from procedural to declarative 
programming find the assignment difficult For those 
students, we have included another optional exercise, 
QUERY.AST, designed as a warm-up for the second 
knowledge engineering assignn)er«t 

The third knowledge engineering assignment extends the 
second by adding database access. It can be accompanied 
by discussion of expert systems components in larger 
systems, interfacing to real-time instruments, and shdls 
such as PC Expert Professional (Software Artistry) 
which make provision for knowledge bases embedded in 
conventional programs. 

Induction* VP Expert also has a simple induction 
c^bility. In the assignment KEINDUCE, the students 
use the same situation as KEl, but solves it by 
induction instead of writing rules. This example is used 
after the knowledge engineering assignments, and it 
illustrates induction and gives the suidents a chance to 
compare induction and rule writing. Hgure 4 shows the 
induction table (decision exemplars) used to generate the 
solutkm knowleidge base. 

Shell Evaluation* The final module covers shell 
evaluation. There are many factors to consider in 
evaluating a shell. We discuss the factors shown in 
Table 1 in the course, and assign reading on shell 



evaluation (Firdman, 1986, Shepard, 1987, Olsen, 
1987, Ciu^nbaum, 1988). Additionally, students are 
given an assigiiment in measuring shell efficiency using 
a benchmarking technique we have developed (Press, 
1989). 

The benchmarking rpproach uses a program to generate 
test benchmarks of a given size. The course materials 
include source and object code for benchmark generating 
programs for the VP E:;pert and Ml shells. Handout 
files with instructions for using the benchmark 
ge4)erators are also included. 

There is a benchmarking assignment in which the 
students are asked to generate some test knowledge 
bases and time their execution. The student also 
experiments to see how large the knowledge bases are 
and how much memcvy they require. 

Conclusion 

The teaching material described above has been tested in 
an MBA course, and revised in accordance with that 
experience. A S6-file disk containing the class handouts, 
homework assignments, knowledge bases, and solutk)ns 
to assignments is available from the author. In 
requesting it please send a formatted disk (3 or S inch) 
in £ self-addressed, stamped mailer. 
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Figure 1: A Listing of FOODl.KBS 
ACTIONS 

DISPLAY "LETS DECIDE WHAT SORT OF WINE TO DRINK WITH YOUR MEAL. 
PRESS ANY KEY TO BEGIN THE CONSULTATION.-" 
DISPLAY **** 
FIND WINE_COLOR 

DISPLAY **YOU SHOULD HAVE {WINE_COLOR) WINE WITH YOUR (MEAL.TYPE).** 

RULEl 

IF ME AL.TYPE = MEAT 
THEN WINE.COLOR = RED; 
RULE 2 

IFMEAL.TYPE = FISH 
THEN WINE.COLOR = WHITE; 
RULE 3 

IF FOOD = HAMBURGER OR 
FOOD = STEAK OR 
FOOD = HOTDOG 
THEN ME AL.TYPE = MEAT; 
RULE 4 

IF FOOD = LOBSTER OR 
FOOD = SHRIMP OR 
FOOD = SALMON 
THEN MEAL.T YPE = HSH; 

RUNTIME; 

ASK FOOD: "WHAT ARE YOU GOING TO EAT?"; 

CHOICES FOOD; HAMBURGER,STEAK,HOTDOG,LOBSTER,SHRIMP,SALMON; 
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Machine Readable Cburse^vare 

Figure 2: Trace of FOODl.KBS 

Goal; FIND WINE^COLOR 

Invoke RULE-1 (because it sets WINE_CX)LOR) 

In v(*e RULE-3 (because it sets MEAL JTYPE) 
seek FOOD 

FOOD = "tobster" (input by the user) 
RULE.3£ftils 

Invoke RULE-4 (because it also sets MEAL JTYPE) 
RULE-4 succeeds, so MEAL^TYPE = Tish" 
RULE-1 fails 

Invoke RULE-2 (because it also sets WINE^COLOR) 
RULE-2 succeeds, so WDME^CCX-OR = "while" 
Display WINE JTOLOR = "white" to the user \ 

Figure 3: Setting for the First Knowledge Engineering Assignment 

You arc a knowledge engineer (KE) assigned lo build an expert system to help recommend data management software 
to people coming into computer stores. Your donuun expert (DE) is the owner of the largest Computerland store in 
town. You interview him to see how he helps customers choose a data management software package: 

KE: How do you decide which data management program to recommend? 

DE: Well, first off I find out whether they need a DBMS or whether they can get by with a file manager. 
KE: How can you determine that? 

DE: I just ask if they have applications where information from multiple files must be brought together. If the 
answer is "yes," I know that they need a DBMS. If it is "no," I know they can get by with a simpler and 
cheaper file manager. 

KE: Isee. Well, what do you do next? 

DE: That depends. If they only need a file manager, it's simple. We carry three file managers: FMl, FM2, and 
FM3FM3 is my favorite but it can only handle files that will fit in memory up to around 1.000 records. If 
their largest file will be under 1 ,000 records, I recommend FM3. 

KE: But what if they have larger file? 

DE: Well, FMl is a powerful program, but it takes some sophistication on the customer's part. If he or she is 
an experienced user, I recommend FMl, but for beginners, I suggest FM2. 

KE: Okay, that takes care of the file managers, but what if the customer has multiple file applications and 
needs a DBMS? 

DE: Well that's pretty straight forward too. We carry three DBMS programs: DBl, DB2 and DB3, ?o it's just a 
matter of choosing one of them. In choosing among them, I go back to customer sophistication. A highly 
sophisticated customer can use DB 1 which is the most powerful. If the customer is experienced, but not an 
expert, I suggest DB2, and recommend DB3 for a beginner. 

KE: Okay, I think I have the idea- Let me work up a prototype version of the program. Ill show it to you next 
week. 

Figure 4. An Induction Table for the Situation in Figwe 3: 
Mult Files Filesize Expertise* Dbase 



no <1000 records ♦ FM3 

no >1000 records beginner FM2 

no >1000records ♦ FMl 

yes * highly sophistk:ated DBl 

yes ♦ experienced M2 

yes ♦ beginner M3 
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Table 1* Factors to Consider in Shell Evaluation 
The User Interface 

• Can the shell explain how it came tc a conclusion? 

• Can the shell explain it is asking something during a consultation? 

• Will the shell diq)lay help to the usei? 

• Arc there flexible prompts and menus for user input? 

• Is there data type checking? 

The Developer Interface 

• Is the KB entered with an external or built-in editoi? 

• Is the internal editor &st and powerful? 

• Are there de'xigging aids? 

• Is there a playback capability? 

• Is there an induction capability? 

The Language 

• Alt certainty factors aUowed? 

• Is forward duuning possible? 

• Can the shell read and write external files and devices? 

• Can the knowledge engineer conUol the consultation sequence? 

• Can you write subroutines and functions in an external programming language? 

• Are frames supported? 

Efficiency 

• How Cust are k>ading and execution? 

• How much disk storage does the KB require? 

• How large a KB can be executed? 

• How laiige a KB can be loaded into memory? 

• Will the program use extended memory if you have it? 

• How fast is development? 

Company Policy 

• Can you create nin-only versions of the finished KB, and is there a license fee? 

• Is the company marketing the shell committed to support and ongoing development? 

• Is there antegrade path? 

• How much does the program cost? 

Larry Press 

California State University, Domingucz Hiils 
107?^ Esther Avenue 
Los Angeles, CA 90064 
(213)475^515 
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THE USE OF COURSE OF ACTION 
TO DEVELOP FIVE UNITS OF INSTRUCTION 
IN A CARDIOVASCULAR SYSTEMS COURSE 



Richard S, Hannah and Peter //. Harasym. University of Calgary 



Abstract 

Course of Action was used to ctx^atc flvc units of 
computer assisted Icaming in cardiology. The material 
was offered to medical students as pan of the planned 
learning experiences. A thorough evaluation of the 
authoring tool was undertaken. The merits and 
limitations of the system will be presented along with 
helpful hints to prospective vscxs. 

Introduction 

In 1986 the new authoring system ca'led "Course of 
Action" was introduced at the Development of 
Computer-Based Instructional Systems (ADCIS) 28th 
International Conference in Washington. D.C. Many 
educators were excited about the potential of reducing 
the time and effort required to develop quality computer 
assisted instructional (CAI) materials. The system has 
new been available of four years; however, there are few 
evaluation reports outlining its advantages and 
disadvantages. This paper shall outLne its use in a 
medical undergraduate course and summarize the 
system's main advantages and disadvantages. 

Courso of Action 

Jesse M. Heiries (1987) describes "Course of Action" as 
the most advanced authoring system you can buy. Its 
power and simplicity support more effective authoring 
procedures and better insmjctional interactions. Some of 
the advanced features include: direct editing, great 
graphics, data-driven animation, talented text, advanced 
answer analysis, concurrency, interactive video, 
individualized branchin[r. elecuonic student notebook, 
models, variables, computer-managed instruction, 
external interface, portability, local languages, and 
automatic documentation. But, from a practical 
prospective what are its true advantages and 
disadvantages? This evaluation is based upon personal 
experience obtained in creating five units of instruction 
in cardiology. The program offers approximately three 
hours of individualized instruction. The CAI materials 
were tested with nrst year medical students as part of ihe 
instruction offered in a course entitled The 
Cardiovascular System. The structure of the CAI 
program is described below. 

Program Structure 

Purpose of CAI materials. The goal of the project 
was to develop and evaluate a CAI program using 
Course of Action. Cardiac malformations from an 
interdisciplinary perspective were selected to test the 
many features offered by Course of Action. The target 



audience were first year medical students but the 
materials were also seen to be suitable for post-graduate 
students. 

The majority of CAI materia^<J available today arc 
discipline based (e.g. the anatomy of the heart, cardiac 
physiology, cardiac radiology). The curriculum in the 
FacuUy of Medicine at the University of Calgary is 
arranged according to body systems and taught using a 
multidisciplinary approach. Thus, the single 
disciplinary CAI packages identified were inadequate 
since they did not integrate knowledge and skills across 
disciplines. 

In designing the CAI course, basic science concepts 
from the disciplines of Anatomy, Physiology and 
Pathology were combined with the clinical science 
disciplines of Cardiology and Radiotogy. This integrated 
information parallels that required to make an accurate 
diagnosis in clinical practice. 

Five common congenital heart defects were selected 
(Auial Septal Def*x:ts, Venurtcular Septal Defects, 
Transposition of the Great Vessels, Patent Ductus 
Arteriosus and Tetralogy of Fallot). For each of the five 
defects, the student is presented with a menu offering 
the following options: 

Nonnal devetopment 

Abnonnal development 
• Diagnostic skills 

Knowledge Quiz 

Each of the above four sections are independent of each 
other allowing the students to choose an individual 
pathway through the materials according to personal 
needs. For example, on the first attempt, the novice 
student may examine all four sections within a given 
congenital heart defect in the above defined order. On 
further attempts, the same student may more 
selective and go into specific sections while bypassing 
others. This arrangement makes it of equal value to 
more senior students including interns arul residents who 
may wish only to "brush up" on their diagnostic skills. 

Since most of the concepts required to master the 
material arc visual and auditory, each defect is presented 
with a minimum of textual information with the 
emphasis placed on animations of heart growth (both 
normal and abnormal), static in.s^es (both schematic 
such as plots and real images such as X-rays), and 
graphic flow diagrams (e.g., to demonstrate pressure and 
oxygen levels in different conditions along with 
corresponding simulated heart sounds). 
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To permit discriminative use, the normal and abttcrr^al 
development sections contain up to six nested levels k^i 
infoimation, with level one representing genera! 

werview" and each higher nested level containing 
progressively more detailed information. The section on 
normal development takes the student through a series 
of logical progressions related to a specific defect on 
how that portion of the heart normally forms. The 
abnormal development section provides the student with 
the information regarding known mechanisms and 
underlying causes of the defect plus the physiological 
consequences of the defect In the diagnosis section the 
student is exposed to various diagnostic techniques 
including electrocardiogrdms, phonocardiology, and 
radiology as they relate to the defect in question. 

The knowledge quiz section integrates the core 
knowledge from the normal, abnorm^, and diagnostic 
sections. This section represents a classic 
questions/answer format tutorial. However, where 
appropriate, all the other sections contain 0[qxHtunities 
for student interaction. For example, in the normal 
development section* students are required to choose 
between several possible scenarios on how a selected 
region of the heart develops. Each scenario is taken to 
its end point, with the student being given feedback of 
where they made logic errors or where they require mcHC 
inf(»ination to continue. 

In the knowledge quiz section, students are also offered a 
patient simulation to test their diagnostic skills. After 
reading a case history of the patient, the student is asked 
a series of questions, the responses to ^ich are added to 
a diagnostic chart. The process continues until the 
student is able to diagnose the patient's problem. 

Analysis 

Direct editing. Changes or additions to courseware 
can be make while being presented in student mode. 
Tools to make changes and additions will appear 
immediately when the edit mode is executed. Changes 
are made directly on the screen and when completed the 
author can re-enter student presentation mode. 

Graphics. The program comes with a resuicted 
graphics package which is primarily useful for creating 
boxes for highlighting text or flow diagrams. The 
majority of images we utilized had to be created with a 
separate graphics package such as MacPaint or digitized 
with a scanner from drawings or pictures. However, 
using the copy/paste function or "import graphics** 
feature which takes you out to a graphic application and 
back without having to exit the authoring system 
makes gr^hic handling a fairly efficient process. 

Data-driven animation 

Talented text. Pull word-processing power is 
available for text presentation. It is possible to position 
text anywhere on the screen. There is a wide variety of 
text styles, text fonts, and text sizes. As with most text 



processors, margins, tabs, and decimal tabs can be 
adjusted. Text automatically wraps around at margin 
settings as you type. 

Answer analysis. There was no need to limit 
students to multiple-choice questions. Extensive 
interaction c^bilities are available: text entries, mouse 
actions, pulldown menus, on saeen push buttons, and 
function keys are available in any combination. 

Time limits for entries can be set Time limits can be 
used to branch or provide students with helpful hints. In 
addition, students can be interrupted and put through 
remediation based upon their accumulated performance. 

Students can be branched according to correct or 
incorrect responses. Since student performance is 
accumulated, they can also be branched according to 
their level of krK>wledge or decision making skills. 

Course of Action imposes rw limit on the number of 
anticipated responses. Since all re^nses are active, the 
system enables response-sensitive actions. This is a 
powerful q}tion if property used. Naturally language 
processing is limited to key words and for most 
implications, this was found to be adequate but in some 
situations restrictive. 

Interactive video. Computer control of video is 
apparently simple. This aspect of the authoring system 
is currently being evaluated. Unfortunately, this option 
arrived too late to be properly evaluated. 

Course Design. Branching options are very powerful 
with an unlimited number and pattern capabilities. 
Branching is achieved by linking icons. There are eight 
course design icons available: display, animation, erase, 
wait, decision, question, cakulation and map. Graphic 
hierarchical presentations of the icons provide a clear 
outline of the course stmcture. This perhaps is the most 
powerful feature of the system. 

The five cardiology units can be seen to be a course of 
significant size (i.e, SOOOK). In a course of this 
magnitude the course structuring would normally have 
been a problem. The use a small number (i.e., eight) 
powerful design icons enabled even the neophyte 
programmer to easily and quickly create sophisticated 
CAI materials. Assembling courses in this manner was 
found to be particularly useful in building and testing 
highly nested course sections. 

Once the course logic was established for one unit, the 
same basic logic was use^t to build the remaining four 
units. The separation of course logic from content saved 
enormous amount of time. This feature was seen to 
enable the development of outstanding interaction 
models which could be transported to other authors or 
courses. It clearly was seen how a library of models 
could be created and shared with other course developers. 
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External interface. Course of Action permits 
running of routines written in other languages (e.g., C 
and Pascal). Data can be passed to and from Course of 
Action to routines. External files can be created, written 
with data, and read as desired. Apparently, selected 
performance data can be transferred into a spreadsheet 
and used to compile performance data. 

Student Performance Recordings. In addition, to 
data transfer capabilities, the authoring system provides 
extensive data collection capabilities through "system 
variables" and "system functions". These capabilities 
can retrieve any student answer, provide extensions to 
the text analysis capabilities, and plot data on screen. 
Thus, the system can easily provide a student with a 
score or other quantitative data for use in a variety of 
ways. However, it was extremely difficult to track the 
student results in the form of a permanent file for 
evaluation of student performance by the author. Since 
evaluation is an integral part of any CAI program, most 
authors require this information which should be an 
integrated part of the system's software- Unfwtunatcly 
with the current method, the manual did not inclu(^ 
examples of how to achieve this performance recording 
and many hours of labour were lost trying to figure the 
procedure that could be used. To date this difficulty 
remains unresolved and is a serious shortcoming of this 
author tool 

Screen Displays. On Macintosh System II 
microcomputers, the authoring system allows for a full 
color display. This feature was found to be an excellent 
method of color coding the lesson screens, which 
constantly reminded students which section of the 
program they were in. In addition, due to the complex 
nature of many of images, color was used to indicate 
specific growth regions in the heart, in lieu of arrows, 
resulting in uncluttered, pleasing pictures and diagrams. 
Although the color feature is useful, the system also 
works well on a monochrome screen. 

Animation, The animation package that is included 
in the authoring system is well documented but limited 
to a naximum of 64 x 64 pixels. Due to the low 
resolution, one is limited to animating simple 
structures or moving objects such as arrows. Although 
these processes are easy to master they did not meet the 
need for animating complex anatomical structures. To a 
certain extent we overcame this limitation by aligning 
the individual cells of the subject to be animated on 
separate screens and running tne entire screens in 
sequence, in a loop. One must note that this method 
uses a large amount of disc space and the speed of 
running the screens is limited by the speed of processor 
in the Macintosh being used. Optimally, it would be 
preferable to link a better animation application to the 
authoring system for those users who require better 
resolution and controlled movement of the images. 

Sound Digitization. The sound digitization package 
supplied with the authoring system utilizes the 
application "Sound Wave". Besides a few "bells and 



whistles" which were used to make it interesting for the 
student, our primary use of the sound application was to 
digitize the heart sounds producer' by the various heart 
defects. Although the sound application provides for a 
wide range of manipulation of the digitized sound, the 
quality of the sound is limited by the built-in 
Macintosh speaker. An external speaker was essential 
for complex heart sounds where students had to make 
fine cardiac discriminations. Perhaps in the future, the 
developers could supply an appropriate ^)eaker as an 
option, along with the other digitizing hardware 
supplied. 

Concurrency and Perpetual Options. The 

concurrency option allows a function to be performed 
with the one which follows. So, for example, a heart 
animation could be started along with the playing of a 
digitized sound. The two ftmctions are performed 
together. This option was found to be extremely useful 
.ind necessary within the course. 

The perpetual (q}tion allows the current ftmction to be 
performed along with all of the following until the 
function is stopped or erased For example, if the main 
menu was to be made available throughout the course, a 
push button labelled "main menu" coukl be perpetually 
active. This featiu'e would be especially useful for a 
glossary of terms. This option was not used within the 
course. 

Computer-Managed Instruction. Special 
capabilities permit Course of Action to jump to other 
non-Course of Action applications (e.g., Lotus 123). 
When the student quits using the software jumped to. 
Course of Action will resume exactly where it left off. 
This feature was not tested 

Portability. Currently, Course of Action has the 
ability to present courseware written on the Macintosh 
oil both the Macintosh and IBM/IBM compatible 
microcomputers. This feature is supposedly automatic 
but was not tested at the time of writing this report. 
This feature will be tested in the near future. 

Local Language. Course of Action is to be made 
available in a variety of languages (e.g., German and 
Chinese). Only the English version was tested. 

Automatic Documentation. The system enables 
complete documentation of courseware design to be 
transcribed to paper. There are many printing options 
available. It automatically annotates the design by 
graphically printing tl\c course flow. The individual 
icons can also be printed along with the content of each 
screen. This ability automatically prints a hierarchical 
schematic of the various course components, saved 
considerable time and improved the course presentation. 
An option of index cross-referencing each icon in the 
course, made it easy to work with the large course 
design. The print options were considered to be an 
invaluable aid. 
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Conclusion 

"Course of Action" is the most comprehensive 
authoring system we have encountered to date. It allows 
a ''content expert" with little computer background to 
design, input, and update a complex CAI program. 
Although this is possible, it is far better if resources 
permit, to work with a pedagogical expert and a 
programmer. A team approach permits the content 
expert is to ^nd more time designing the materials 
necessary to produce quality CAI experiences for 
students. 

Although there are many fine features available in the 
authoring system, the three most positive features 
include ease of course design, creation and 
documentation. All the features of the system have been 
carefully designed and, in our opinion, would meet most 
CAI needs with medical education. However, a serious 
shortcoming is the lack of a switch to automatically 
turn on student performance recordings and we assimie 
that Autborware would overcome this limitation in the 
near future. 

Many of the difficulties and limitations that were 
encountered related to using a Fat Mac with lessons in 
excess of SOOOK. This was seen to be a hardware 
limitation and not a system limitation. Even by 



generating the individual heart defect lessons as separate 
entities and amalgamating them fcM- the Hnal student 
version, we found that more than I meg of memory was 
required fcM- most operations. In addition, utilizing the 
authoring system without a hard drive, even for writing 
or running the smallest of lessons proved impractical. 
One or two 800K floppy drive systems leave little room 
on the disk for the lesson. Thus, a Macintosh System II 
microcomputer is highly recommended. 

In conclusion, we highly recommend Course of Action 
as an authoring system for those interested in 
developing large, complex CAI learning experiences. 
This system, in our opinion, is the best available to 
date. 
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AN INTERACTIVE VIDEO TEACHING SURGICAL SKILLS 



Richard S. Trynda, University of Colorado Health Sciences Center 



Abstract 

This paper will discuss the design and development of 
an InfoWindow module on Emergency Surgical 
Procedures including the design, instructional strategies, 
algorithms and authoring system used 

Introduction 

This project was undertaken by the Office of 
Educational Services at the University of Colorado 
Health Sciences Center, and supported by the IBM 
Corporation. Content was provided by the Department 
of Emergency Medicine. InfoWindow was selected as 
the delivery system (hardware platform). 

Design 

The single screen interactive video format allowed a 
screen design which consisted of a video insert in the 
upper left comer occupying approximately 4/5th5j of the 
screen. Surrounding the video insert is an overlay upon 
which user control/input options, text and feedback are 
displayed This design allows consistent placement of 
user control/input options while simultaneously 
viewing the unobstructed video. User input and 
selection of control options are accepted via the touch 
screen. 

Interactive video is a natural delivery medium for 
achieving objectives incorporating visual recognition 
skills such as the following. 

• Given a tray of equipment used to perform 
thoracotomy, cricothyrotomy and related 
procedures, you will be able to identify the 
equipment by name. 

Cognitive psychology suggests presentation of 
information and skills in a context similar to that in 
which the information and skills will be used. The 
instructional strategy selected for this objective is a 
mixture of drill and practice, discovery learning and 
gaming. User interaction begins with a video still frame 
of a surgical instrument tray is displayed in the video 
insert. The user is asked to select by touching an 
instrument when given its name. User options are 
"return to previous menu" and "help". Help allows the 
user to touch an instrument on the tray for more 
information using a hypertext/discovery learning 
approach. Displayed counters indicate the number of 
instruments correctly selected and the number of 
attempts to correctly select an instrument, incorporating 
a gaming strategy. 

The algorithm for implementing this instructional 
strategy used a database and a looping structure. The 
database was constructed containing instrument names. 



touch area screen coordinates, and feedback, both 
positive and corrective. The looping structure, using a 
random number, control program execution and user 
interaction with information fipom the database. 

Interactive video is also a natural medium when an 
objective incorporates components of psychomotor 
skills such as the following. 

Using computer software and a videodisc, you will 
be able to correctly perform a simulated 
cricothyrotomy, thoracotomy, perocardiotomy and 
thoracic aortic cross-clamping. 

Again cognitive psychology suggests that pr2CtlcQ is a 
critical component to the learning of a psychomotor 
skill. In establishing an instructional strategy, a task 
analysis of cricothyrotomy, thoracotomy and related 
surgical procedures was performed. The resulting 
procedural knowledge for each prr :edure created a 
database which was sequential, based on action, and 
contained information on the required instrument, 
matenal, orientation and extent for each action. From 
this database, an instructional strategy was designed 
using a practice format with positive and corrective 
feedback. The structure of the database arising from the 
task analysis suggested a n'.enu system using overlaying 
windows. User interaction begins with a video still 
frame of the surgical field is displayed in the video 
insert As the user selects an action, instrument, etc, the 
previous selection is maintained on the screen. The 
ability to change the previous selection is provided. 
After an action and all related items are selected, both 
text and video feedback are provided Textual feedback is 
either positive or corrective. Video feedback contains a 
motion segment of the appropriate surgical action at 
this point in the procedure. 

The algorithm for implementing this instructional 
strategy used a database and a sequential code structure. 
This structure called nested units generating overlaying 
windows from information in the database. 

Development System 

TenCORE was selected by the software 
designer/developer as the authoring system because of 
familiarity with the structure/syntax of the language, its 
flexibility, its ability to communicate with InfoWindow 
software and the use of both vector and bit-mapped 
graphics which yields compact executable code. 

The surgical procedure practice section of the 
instructional module was extracted to create an 
assessment module. Data was collected during a field 
test using this module. Results from the field test will 
be presented if available. 
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USING COMPUTER SPREADHSEETS FOR INSTRUCTION IN 
COST CONTROL CURRICULUM AT THE UNDERGRADUATE LEVEL 



James J. Buer germeister. University of Wisconsin-Stout 



This paper discusses how the instruciional use of 
spreadsheets can help focus student attention on the 
influence of cost control variables and provide increased 
practice opportunities for students learning cost control 
concepts. Discussion addresses the question of whether 
any instructional improvements would justify increased 
use of computer spreadsheets for leaching content in 
undergraduate cost control courses. 

NOTE: This work represents a small portion of a 
comprehensive study on the effects of computer 
spreadsheets on cost control concept attainment to be 
published in its entirety in the near future. 

Inttoduction 

The microcomputer has dramatically affected twenu^u. 
century education. Microcomputers offer educators 
unique opportunities for presenting curriculum and for 
addressing research questions about motivation, 
achievement, and long term consequences of this new 
technology (Leppcr and Chabay 1985; Roblyer 1985). 

The ability to use and understand computing is 
becoming as important as our ability to understand and 
handle the written word. A computer literate population 
is as necessary to an information society as raw 
materials and energy are to an industrial society 
(Deringer and Molnar 1982, p. 3). 

Not ail educators would support the above quote in full; 
however, the basic premise is generally accepted 
throughout the educational community (Mynatt, Smith, 
Kamouri, and Tykodi, 1986). As microcomputers 

Figure 1.01. Computer Use and Student Control Continuum 



continue to pervade the classrooms of the country, the 
magnitude of impact is less clear. Therefore, education 
must address three questions: 1) What is the true impact 
of using computers in the classroom with respect to 
achievement, cost-effectiveness and unanticipated side- 
effects? 2) What curriculum content should be taught 
with computers? 3) How should this be accomplished? 

Computers today represent a powerful tool for 
education. The computer of the year 2000 will be more 
powerful, changing the life of both teacher and student. 
"But it is not the tool that gets the job done, the person 
using it does,** (Luke, et al, 1986, p. 8). Computer 
spreadsheets allow for the use of real-world problem sets 
and at the same time permit the student to work on 
concepts associated with the curriculum content as 
q)posed to the mechanics of the problem. Furthermore, 
spreadsheets enable the user to capture and save the 
entire concept for future scrutiny and review; they 
provide the user with an array of feedback options. 

Computer Software 

Traditionally, most computer use in schools has been 
concenu^ited in two areas: pure programming skills such 
as BASIC or COBOL; and orientalion/tutor.al and 
practice programs commonly referred to as computer 
assisted learning or CAL (California State Deparunent 
of Education 1985). Figure 1.01 suggests a continuum 
between CAL software programs and formal 
programming users as a function of control. 
Application software lies between the two traditional 
approaches offering the user structured interaction.. 
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Unlike CAL software, which commonly drills students 
on particular content matter such as vocabulary or 
mathematical computation (drill and practice), or 
presents information and suggest where the student 
should go next in the program (tutorial), spreadsheets 
provide an alternative dimension. Spreadsheets are tools 
designed to organize data, compute numeric variables 
and present solutions to real problems in a timely 
manner. The **toor school of thought maintains that 
the computer is one of many tools devised by humans 
to help them perform certain tasks. What is critical is 
for the individual to use the computer to meet his or her 
need(Megarry. 1983). 

Secondly, spreadsheets are used to illustrate financial 
models and other related applications through the use of 
data queries and sorts. This characteristic can be further 
exploited instnictionally with the use of a special 
overhead projector, providing students the opportunity 
to see the effect of different numeric variables on model 
output In essence, spreadsheets provide educators with a 
wide array of options to help promote learning fmancial 
concqxs. 

One of the single most important reasons for using 
application software for teaching is the amount of 
additional time a student may then have to spend on 
content-related problems. Ease of calculation provided 
by spreadsheets makes possible many practice 
opportunities related to numerical analysis. In addition, 
these plication programs, such as q}readsheets, may 
also provide opportunities for the development of 
critical-thinking and problem-solving skills (Eastment, 
1986). 

Ironically* few students in the public schools use 
spreadsheets and data bases as learning tools in 
appropriate courses. Lambrecht (1986) found that 
business teachers in Minnesota avokl the computer for 
the teaching of content Her flndings reported an average 
of three topics out of a possible 19 were taught using 
the computer. Reasons for teachers not using 
application software fot teaching related content in the 
classroom might be linked to the lack of available 
hardware. Furthermore, empirical research designed to 
address questions about computer spreadsheet 
effectiveness, appropriate curriculum content and 
methodology is very scarce. 

Mori on Computer Spreadsheets 

An illustration of computer technologies that can 
augment mental functioning is the electronic 
spreadsheet, stjch as Multiplan, Quattro and Lotus 1-2- 
3. Several million copies have been sold since they 
appeared in 1979 (Pea, 1985. p. 167). Elecuonic 
spreadsheets are software programs for micro-computers 
which serve as powerful tools. The screen images 
physically resemble paper ledger sheets, with cells 
organized in rows and columns. However, in an 
electronic spreadsheet, one can place a label, number, 
calculation, or a formula in any spreadsheet cell, which 
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can subsequently be edited, copied, or moved. The 
results of calculations in the formula area appear as the 
content of the cell. The most dramatic difference from 
static paper spreadsheets is that one can change cell 
entries and see the repercussions of that change 
recalculated immediately throughout the spreadsheet 
(Levy, 1984). This what-if property has dramatic 
consequences for the development and use of budgeting 
and cost control models. 

The typical spreadsheet also offers some limited 
programming language capabilities, including thv^, 
ability to manipulate strings, write macros and to do 
interactive calculations. In addition, built-in functions 
can be combined to produce a plotting capability, 
thereby providing both a graphical and numerical 
method of illustrating relational concepts. Although the 
q)readsheet program requires that the user be able to 
q>ecify the relationships to be explored, it does not 
require that the end-user be a programmer (Hewett, 
1985). 

Similariy, providing some students with spreadsheets as 
tools to experiment with cost control concepts vs. a 
lecture-discussion format without spreadsheets should 
result in different outcomes because of the unique 
activities associated with using spreadsheets. 

Problem Statement 

What is the effect on understanding cost control 
concepts when computer spreadsheets are used to: 
display cost control models, change key numeric 
variables within the cost control concept, provide 
multiple examples of concept use, allow active learner 
participation in model development and evaluation of 
spreadsheet feedback in contrast to using 
lecture/discussion methods to present and practice the 
same content? 

Purpose of the Study 

The purpose of this paper is to discuss the important 
issues surrounding the use of computer spreadsheets for 
the teaching of undergraduate cost control concepts. 
Methods for using the computer spreadsheet will be 
explained and supported with information processing 
theory. 

Need for the Study. Some of the more critical 
unanswered questions in education today deals with 
computers. Specifically, do computers significantly 
improve instructional methods and consequently student 
performance? Literature in the field contains numerous 
articles pertinent to computer assisted instruction (CAI) 
or computer assisted learning (CAL). However, there is 
no empirical research to support the use of implication 
software (spreadsheets) for the teaching of content. 
Robert Gagn6 offers the following thoughts on the 
subject of computers and instruction. 

The computer remains a highly intriguing way 
of presenting insuiictional content, and of 
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taking some account of the learner's response 
to content Kfaybe learners simply prefer to 
look at screens rather than pages. Maybe they 
like the pressing of keys as a kind of concrete 
action that is not demanded by a page of 
printed text Maybe the reassurance of a visual 
message di^Iaying an answer to a question is 
inhevently more pleasing thanlodcing up the 
answer on a printed page (1982» p 3). 

The computer spreadsheet is a powerful tool, it i»ovides 
the user with numerous opportunities to experiment 



with and evaluate information/data, u used 
appropriately* spreadsheetw can offer educators new 
variables in the area of instructional methodology. By 
granting students the use of spreadsheets to process, 
apply, and evaluate cost control concepts, it is possible 
to influence the events of learning (see Figure 2.01) as 
outlined by Gagn6 and Briggs (!974). Identifying which 
instructional event(s) the q>readJ)eet is influencing, and 
why, would be helpful for designers of instruction. If 
learners do prefer screens to written pages or pressing 
keys instead of using a pracil, educators need to know 
the degree (^impact this has on learning perfonnance. 



Figure 2.01. The Informati on-Processing Model (Gagnd, 1977, p.S8) (Spreadsheet events added by the author.) 
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Understanding i^^ specific outcomes of application 
software on student performance in relation tc the 
events of instructions will help educators address issues 
surrounding: 1) the role education should play in 
preparing students to use application software as tools 
in appropriate curriculum; 2) the best way to integrate 
application software with content; and 3) the impact of 
application software feedback on student performance on 
concept attainment 

Assumptions 

The following assumptions are made: 

1 . The external events and dynamics of a learning 
situation influence learning. These include: 
availability of stimulation to activate 
attention, variety and type of cues available for 
student perception, suggested schemes for 
coding information, suggested schemes for 
remembering^etrieval, opportunities to transfer 
learning to a new situation, actual student 
peifonnance and idnfoccement/Teedback. 

2. The use of computer spreadsheets as tools 
provide unique external events with respect u> 
how inforniation is processed. This is 
attributed to the spreadsheets* capability to 
stimulate student attention; promote schemes 
for coding and retrieval/storage of infomiation; 
permit the student to develop, apply and 
manipulate a concept in its entirety-saving the 
work for easy retrieval and review; allow the 
user to change numeric variables-resulting in 
unlimited what*if testing; and provide the 
student with timely evaluation of spreadsheet 
feedbadL 



Nature of instruction 

To reach the objectives of this study two insmictors 
taught one experimental group and one conuol group 
each. The population of each was set at 4i , for a tot^ 
of 82 subjects. 

For the experiment, all classes met in the same building 
utilizing a traditional classroom for the control groups, 
and the computer labonitory for the treatment groups. 
This facility is equipped with 21 IBM PS»2/Model 30 
computers utilising a local area networic (LAN). This 
system uses the PS-2/Model 60 as a file server, with 
each individual PS-2/Model 30 workstation having the 
ability to access programs loaded on the file server. The 
Quattro spreadsheet was used for the experiment 

The experimental groups (spreadsheets) were introduced 
to the Quattro spreadsheet through a six lesson uitorial 
program, published by the software company (Getting 
Started with QuatUo, 1987). Two weeks of instructional 
time was allocated to orientation and instruction on how 
to use the spreadsheet 

Instruction in both the experimental and conurol groups 
was accomplished by the insuructor introducing cost 
control concepts via the computer. A special overhead 



projector (SHARP QA-50 Computer Projection Panel) 
was used to display information from the insmictor's 
computer monitor onto a large waU screen, allowing the 
entire class to see a functional spreadsheet perform. In 
addition, each student was provided with a hard copy of 
the spreadsheet model to serve as resource and provide 
step-by-step instructions on how to complete the 
concept assignment 

All subjects were introduced to the concepts by the 
instructor demonstrating each one of the cost control 
models on the computer and a QA-SO (special overhead) 
projector. In addition, the student was provided hard 
copies of the concept lesson and a cost control 
textbook. 

In the experimental group, the students were provided 
with computers to allow for the building of their own 
model worksheet using the spreadsheet at their work 
station, or calling up a partially completed worksheet 
and completing the model by entering the necessary 
formulas and formats to make the qxeadsheet operative. 
Subjects could compare their own spreadsheet model to 
the instructors by utilizing the comixiter file server. The 
student had regulated access to a collection of cost 
control concepts on the file server. 

Control groups wu-e insuiicted through an identical 
method. The instructor demonstrated the cost control 
concept using the computer spreadsheet and presented 
this material via the QA-50. However, the control 
group did not have use of a personal computer at their 
workstation to serve as a resource for work and practice 
on cost control concq)ts. 

The researcher believes the external conditions which 
contribute to learning in a uaditional lecture-discussion 
spproach will be considerably different from the ehemal 
conditions present when computer spreadsheets are 
available for each student to serve as a tool. 

A close examination of two environments (one with 
spreadsheets-one without) identify how and why the 
environments are different as well as alike. Specific 
learning phases include: the stimulus; feedback; and 
opportunity for practice. The spreadsheet offers users 
external conditions which encourage additional 
experimentation and can possibly draw out a greater 
response from sUidents with ^neadsheet availability vs. 
those without. Table 1.01 compares and conuasts the 
role spreadsheets play in the learning phases and relates 
it to the insmictional events as outlined by (Gagnd , 
1977). 

Summary 

The instructional use of spreadsheets can improve 
educational outcomes in undergraduate cost control 
curriculum through: 

1 . The Tool Metaphor, Provide the learner with a 
powerful tool that will allow for increased 
practice opportunities, serve as a contemporary 
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lool to apply cost control concepts and offer to students— increasing the chances of 

"what-ir testing scenarios. identifying the (Vtimum method, 
2. The Medium and Method. Serve as an efficient 3. Teaching Valuable Technical Skills. Expose 

vehicle to transport educational concepts from the student to the technical skills associated 

teacher to students via special technology with •^rsonal computers and application 

allowing for the demonstration of cost control soft' teaching them how to use computer 

concepts using real data. The computer icchw^^ogy in the work environment they plan 

spreadsheet medium will allow the instructor to pursue as a career, 
to use a variety of methods to present material 



Table 1>0L Instructional Events and Associated Learning Phases 



THEINSTOUCnONAL 
EVENTS 


LEARNING PHASES 
ASSOCIATED WITH EXTCRIMENT 


Example of how a concept was presented for learning in 
the experimental group (spreadsheets) and the control 
group (no spread^eets). 




All students participating in the study received a copy of 
the major concept to be discussed This resource included 
the following information and characteristics: 

a) The cost control concept was provided in wriuen 
form. This model concept was produced by a 
computer spreadsheet and did depict actual data under 
two different sets of conditions to help the student 
sec a comparison and contrast of the key variaoles, 
and how this influences the end results. 

b) The model concept did include the following 
attributes: description of the cost control concept, 
how the concept is used in the cost control area, and 
pertinent formulas associated with the concept. 

c) The model concept also included references to rows 
and columns as found on a working spreadsheet 


1. Activating motiyation: The individual strives to 
achieve the goal of learning cost control concepts, 
and in the experimental group the use of a computer 
^xcadsbeet 

Differences between treatments: The expectancy of 
learning computer q>readsheets in the elemental 
group coupled with heightened motivation. 
Implication: Heightened motivation resulting 
from the novelty associated with computers and 
individual student expectancy during early stages of 
the experiment, however, this expectancy is likely 
to decrease over the 16 week study and is unlikely 
to have a significant effect on cost control concept 
attainment over the long run. 


Students in both groups were inuoduccd to the model 
concept by the instructor using a lecture-discussion 
approach. 


2. Informing learner of the objective: Communicating 
to the leanner the desired behavior or performance 
they will be c^ble of as a result of the lesson. 
Differences between treatments: None 


Students in the experimental group and control group did 
witness a demonstration of the cost control concept on 
the instructor's computer spreadsheet; the screen image 
was displayed via a special projector on a large wall 
screen. All students witnessed how the model works. 
Actual data was manipulated using a ^'what-if" 
orientation. Lecture-discussion focusing on the cost 
control concept will follow. 


3. Directing attention: This event in the lesson is 
directing attention to the stimuli which are an 
inherent part of the learning task. In this study both 
control group and experimental group will witness 
the same demonstration. 

Difference between treatments: None 

4. Stimulating recall: This event is designed to help 
the learner access previously learned capabilities. 
Differences between treatments: None 


Students in the control group spent lime developing a 
model of the same cost control concept, however, this 
group utilized the chalk board instead of the computer 
spreadsheet to build the model. Students in the control 
group did have access to lecture notes, textbook, and 
other class handouts to scr/e as resources-as well as a 
souitc of feedback. 


5. Providing learning guidance: These events form a 
pan of instruction during this learning phase. The 
amount of learning guidance provided (the length 
and complexity of the communication or other form 
of stimulation) will vary depending on the content 
and type of students involved in the learning 
process. 



ERIC 



Spreadsheets for Cost Control Instruction 



Table LOL Instructional Events and Associated Learning Phases (cont'd' 



THEINSTOUCnCWAL 
EVENTS 


LEARNING PHASES 
ASSOQATED WITH EXPERIMENT 


Students in the control group spent time developing a 
model of the same cost control concept, however, this 
groiq) utilized the chalk board instead of the computer 
spreadsheet to build the model. Students in the control 
group did have access to lecture notes, textbook, and 
other class handouts to serve as resources-as well as a 
source of feedback. 


The most common characteristic to be sought in 
learning guidance is the orientation to the objective. 
In whatever form it is given, whether as verbal 
statement, hints, diagmms, pictures, or spreadsheets 
the purpose is to insure a form of encoding which 
will enable the learner to recover what he had 
learned and di^lay it as some kind of performance 
at a later time. 

Considering the outcome of cost control concept 
attainment, (the ability to i^ly the concq)t to a 
realistic situation) the spreadsheet provides the 
student with a unique opportunity to actually build 
the idea on a electronic tablet, demonstrate the 
finished product, receive feed back on some numeric 
fonnulas and develop a variety of personalized cues 
for future reference. This difference suggests a 
definite advantage regarding learning and overall 
performance for the experimental fETouo. 


A cost control coocepi problem was distributed to both 
experimental group and control group. Students were 
required u> answer questions regarding the cost control 
concqH. 


6. Enhancing retention: Instructional provisions for 
enhancing retentkxi and receipt of feedlu:k take the 
form of spBCtd reviews. This stq) will preface the 
distribution of a review problem distributed to both 
groups. 

Difference between treatments: None 

7. Promoting transfer of learning: Promotion of 
transfer is brought about by instruction which 
provides novel tasks for the student, spaced over 
time, and calling for the use of what has previously 
beeii learned. This phase often takes the form of 
solving a problem or designing an investigation for 
a given problem. 

Difference between treatments: The experimental 
group will have the use of the q)readsheet tool to 
assist in getting at the answer, building the control 
model, or predicting a outcome. Using the notion 
that the spreadsheet tool is a "amplifier of 
cognition,** the user will have the advantage of 
being able to perfonn more problems in less time. 
Implication: The experimental group will benefit 
because of the increased exposure to practice 
problems afforded by the spreadsheet. The 
^readsheet is likely to improve the transfer phase 
of learning because ui increased activity. 


Both groups participated in a review and discussion 
activity focusing on the cost control concept F^back 
was provided in the form of correct answers (attached to 
the original model) and a presentation of the model using 
the instructors computer and special screen projector for 
all students to see. In addition, students in the 
experimental group had access to a completed model 
accessiMe fifom computer file server. Group discussion 
explained the outcomes and identified the correct 
respoiise(s). Each student was able to compare their work 
to the correct copy displayed on the screen or printed on 
the handout 


8. Eliciting the performance; providing feedback It is 
important to provide an occasion for the di^lay of 
the performance by the student Having learned, the 
student needs **show what he can do,** not only 
for the teacher's purpose, but for his own learning. 
This is referred to as informative feedback. 
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THEINSTOUCnWAL 
EVENTS 


LEARNING PHASES 
ASSOQATED WITH EXPERIMENT 


Both groups were administered a concept quiz after the 
review and discussion period was completed. The concept 
quiz measured how well the subject understood the 
concept (numucr correct) and the amount of time (in 
minutes) it look to complete the quiz. The experimental 
group used the computer spreadsheet to work through the 
concept^ while the control group relied on the hand 
calculator. 


Implication: Students in the experimental group 
are likely to check their work against their 
spreadsheet model for accuracy and correctness more 
often then the control group students. This is 
explained by the limited capabilities of static p^pet 
copy information (control group) vs. a dynamic 
qxeadsheet model-cif)abIe of recalculations, and in 
some instances-alerting the user to errors in logic 
and formulas. 
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Abstract 

A training model is prc^)osed iv^ assisting new users of 
computer conferencing tc utilize effective 
communication strategies while dtbating electronically 
by means of an network. Five pi\rameters suggested 
include computing prerequisites, technical skill 
development, communication strategy expansion, 
human sensitivity training, and study/research skill 
utilization. 

Computer conferencing reflects a process which may 
require individuals to adapt previously acquired 
communication skills in order to utilize electronic 
communication effectively. The preparation of 
individuals to assume active roles within an integrated, 
computer-based network appears to be a consideration of 
growing importance if organizational administrators are 
to take advantage of the potential of new emerging 
technok)gy. The intent of this research was to develop 
an experiential training model for preparing individuals 
to laUiie computer conferencing ^ecdvely. 

Study Sample 

The participants in the research expenment consisted of 
24 undergmduates, majoring in the field of economics, 
at the University of Delaware in Newark. The specific 
course in which these individuals registcied was labeled. 
Consumer Laws and Regulations, which merited 3 
undergraduate aedits upon its completion. The sample 
consi^ of S males and 19 females. Each student had 
completed an introductory course in business computing 
applications by the end of the Sophomore year. The 
sample included 13 juniors and 11 seniors. Only one 
individual in the sample indicated prior experience 
participating in a computing networic at the initiation of 
this experiment No one in the sample had any prior 
experience utilizing computer conn a problem-solving 
capacity. 

Resaarch Design 

The data collected resulted from an experiment which 
was conducted over a four-week period between 
November 17, 1989, and December 15, 1989. In 
addition, a one-week training period preceded the actual 
experiment 

Software and Hardware. The research experiment 
conducted at the University of Delaware used the 
COI A1SNE conferencing software as the mechanism for 
networking the 22 students into a problem-solving 
project An IBM 3090 nuunframe at the University was 
the host machine to which the students used personal- 
business computers from a variety of locations on 



campus. The personal-butiness computers were 
computing terminals which were hard-wired to the 
mainframe, thus not requiring the need for any floppy 
disks by the student. 2^ith and IBM computing 
terminals were utilized by the students when 
keyboarding contributions and re-plies during the 
conferencing project The CONVENE conferencing 
software is an application's program which permits 
individuals to communicate with one another in a group 
format The software package makes use the BITNET as 
the mechanism for enabling individuals to communicate 
asynchronously by means of electronic keyboards 
attached to computing terminals. 

Research Methodology. The experiment was 
carried out in two phases: (1) a training model 
implemented over a one-week period; and (2) a four- 
week computer conferencing ej^ieriment 

During the initial one-hour training session, the process 
of logging on and off the IBM mainframe was reviewed 
in addition to the editing process for making corrections 
as messages are keyboarded During the second one-hour 
session, the undergraduates were introduced to 
CONVENE and guided in its use as a conferencing tool. 
The third one-hour session permitted the under-graduates 
to practice using CONVENE and to actually begin key- 
boarding their messages to one another as the problem- 
solving project began. 

The problem to be solved by the suidcnts was presented 
to them by means of CONVENE conferencing software. 
The undergraduates were guided to access three messages 
which provided the parameters for their study. The 
students were requested to take one of four positions 
related to the marketplace issue of product liability, 
defend their positions among one another, and then 
submit a final statement to their instructor using no 
more than 1000 words. During the initial two weeks of 
the experiment the undergraduates were required to 
identify at east five weaknesses in their position by 
interacting with other classmates-and submit this list 
to their instructor. During the final two weeks, each 
participant was requested to develop a credible defense 
for their position on product liability by seeking 
comments, etc., that assisted them in responding to 
each of the five weaknesses previously identified-and 
keyboard th;>t statement(defense) to their instructor by 
electronic mail at the conclusion of the experiment. 

All students were assigtted secret code labels that were 
not to be shared with cacH oilier. These labels consisted 
of names of various countries throughout the world. 
Therefore, students did not know the identity of those 
individuals with whom they were communicating. AH 
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conferencing was conducted outside of normal class 
routines— and no discussions related to computer 
conferencing and the problem-solving project occurred 
between faculty and students within a classroom setting 
during the experiment. 

Two sets of questionnaires were utilized for gathering 
data from the study sample. Each set of questionnaires 
adopted a 5 point Liken-type scale in which participants 
were requested to indicate their percq)tions of both the 
training program and experiences encountered during 
computer conferencing project 

Version 5 of the computer-based SAS System (an 
acronym for Statistical Analysi:> System) was utilized 
for analyzing data collected. A T-Test (an analysis of 
variance measure) was per the data gathered related to 
confidence levels of participants at the conclusion of 
training received and the degree of difficulty of problems 
encountered during computer conferencing. These 
statistical procedures were performed using the IBM 
3090 mainframe at the University of Delaware. 

Findings 

Data analyzed consisted of responses recorded on two 
sets of questionnaires by panicipants reflecting their 
perceptions related to 12 competencies which were 
introduced and practiced during a one-week training 
program and then utilized during the four-week 
experiment on computer conferencing. The purpose of 
this analysis was to determine if there were any 
significant differences between the perceptions of 
individuals as related to the degree of training received 
and actual post-training technical problems encountered. 

n comparing the mean scores of perceptions recorded by 
study participants for each of 12 competencies at the end 
of the one-week training session and at the conclusion 
of the experiment (as indicated in Table 1 on next page) 
it appeared that there was a difference in the perceptions 
of participants as related to training program 
accomplishments and actual difficulties with the 
technical aspects of computer conferencing. There were 
only 4 competencies during the training program which 
participants indicated a rather high confidence level for 
performing: that is, having a mean rating of 3.75 or 
higher on a 5-point rating scale.. These four included 
logging on (4.54). logging off (4.45). using the PF 
legend function keys (4.14). and contributes (3.91). 
However, there were eight competencies in which 
participants had very little difficulty performing at the 
computer keyboard: that is. difficulty mean ratings of 
3.75 or above on a five-point rating scale, indicating 
only a very slight problem. These 8 competencies 
included logging on (4.91) logging off (4.82) using PF 
legend function keys (4.45) contributing comments 
(4. 19) reviewing old comments (4.19) searching for new 
comments (4.05) using special CONVENE keys (3.95). 
and accessing conference (4.05). 



Table L Mean Scores of Confidence Levels of 
Participants at the Conclusion of Training Compared to 
Degree of Difficulty as Related to Technical Problems 



Competency 


Training 
Confidence 
Level (1) 


Conferencing 

Technical 
Problems (2) 


Logging off 
ACSVM 


4.54 


4.91 


Logging on 
ACSVM 


4.45 


4.82 


Using PF 

L^end 

Functions 


4.14 


4.45 


Contributing 
Comments 


3.91 


4.19 


Using Electronic 
MaU 


3.73 


3.73 


Reviewing Old 
Comments 


3.55 


4.19 


Searching New 
Comments 


3.36 


4.05 


Using Special 

CONVENE 

Keys 


3.32 


3.95 


Accessing 
Conference 


3.23 


4.05 


Getting 

Help/Solve 

Problems 


2.96 


3.55 


Editing 
Comments 


2.90 


3.36 


Printing 
Comments 


2.64 


3.50 



Note: Rating scale for difficulties encountered in 
Column 2 was recoded from original tallies to permit 
analysis of matching perceptions. A rating of 5 
indicates great confidence for data appearing in Column 
(1) and no difficulty encountered in data in Column (2) 
above. 

ThCieforc, it appeared that the degree of confidence at the 
end of the training program did uqi match i5ie degree of 
difficulty in the kinds of technical problems being 
encountered during the computer conferencing 
experiment. In each instance except for the one 
competency of using electronic mail, the level of 
difficulty encountered was considerably less in degree 
than that associated with the confidence level at the 
conclusion of the training program. 

The differences in perceptions between the training 
model experience and the difficulties encountered during 
the computer conferencing experiment appeared to be 
significant Using a Paired T-Test statistical measure. 
Table 2 on the next page re for 10 of the 12 
competencies, the mean scores of the differences of 
perceptions were not equal. Therefore, there appeared to 
be a significant difference between the degree of 
confidence levels held at the conclusion of the training 
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program and the degree of difficulty encountered when 
performing those same competencies during computer 
conferencing. 

Table L Paired-Comparison T-Test of Mean Scores 
of Differences Between Confidence Levels at the 
Conclusion of Training and Difficulty Levels During 
Computer Conferencing (N = 22) 



Competency 


Mean 


Std. Error 
of Mean 


T 


Logging on ACSVM 


0.364 


0.124 


2.94» 


Logging off ACS VM 


0.364 


0.124 


2.94* 


Accessing Conf. 


0.818 


0.252 




Rev. Old Comments 


0.636 


0.168 


3.78* 


dcarcn. r^cw 
Comments 


U.OoZ 




3.3»" 


Printing Comments 


0.864 


0274« 


3.16* 


Contributing 
Comments 


-0.682 


0.266 


-2.56*' 


Using PF Legend 
Functions 


0.318 


0.212 


1.50*' 


Using Special 
caWENE Keys 


0.636 


0.203 


3.13* 


Editing Cbmments 


0.455 


0.215 


Zll** 


Using Elect Mail 


0.000 


0.208 


0.00 


Getting Kelp/ 
Solve Problems 


0591 


0.269 


2.20*' 



'Significant at the .01 Level. 
^ Significant at the .OS Level. 



Computer Conferencing in Consumer Economics 

For 8 of the 10 competencies listed in Table 2 (except 
for editing comments and getting help to solve 
problems) these findings were significant at the .01 
level. 

Conclusions: 

Parameters of Proposed Model 

In establishing a training model for assisting new users 
of computer conferencing to utilize effective 
communication strategics* five parameters were 
identified: computing prerequisites; technical skill 
development: communication strategy expansion: 
human sensitivity training: and study/research ^kill 
utilization. A two-phase training model is imposed to 
permit new users to not only become acquainted with 
computer conferencing but to receive sufficient practice 
before entering into the formal problem-solving 
I^ocess. The resulting proposed three-week training 
model Includes a one-week introductory session 
followed by a two-week guided practice session in 
which a trial exercise is completed enabling users to 
practice questioning skills. It was also concluded that 
trainers must be careful not to assume that new users of 
computer conferencing v.ill auton^iatically transfer prior 
problem-solving skills practice in non-computer 
sunoundings to computer environments. 

Dr. James L. Morrison 
College of Human Resources 
University of Delaware 
302 Alison Annex 
Newaric,DE 19716 
(302)451-8541 



223 



HOME ECONOMICS TEACHER EDUCATORS PERCEPTIONS OF COMPUTER 
COMPETENCE AS COMPARED TO 
OTHER VOCATIONAL TEACHER EDUCATORS 



Cecelia Thompson, Dale E. Thompson, andLori M. Shimomura 



Abstract 

The purpose of this study was to compare home 
economics teacher educators self-raled level of computer 
competence to those of agriculture, industrial, and 
business teacher educators. The study also determined 
vocational students* use of computers and suggestions 
of what was needed to improve computer competence. 

A random sample of 191 home economics, agriculture, 
industrial, and business teacher educators was selected 
from professional directories for participation in the 
study. One teacher educator in each vocational area was 
selected from all states with vocational programs. 

A questionnaire devel(^)ed by the researchers was used to 
gather data. The questionnaire items examined the areas 
of use of computer hardware and software, level of 
competence, what is needed to improve personal 
computer competence, use of computers by vocational 
students, and what is needed to improve student's 
competence. The demographic variables included age, 
years of e?q)erience, gender, computer training, and type 
of computer used. A total of 109 questionnaires were 
returned from 20 home economics educators, 26 
industrial educators, 39 agriculture educators, and 24 
business educators. 

All vocational groups showed similar levels of 
computer competence and use. Most ranked themselves 
fair to good on levels of computer competence. Most 
used the computer and printer for word processing. 
Almost all vocational educators were self-taught and 
perceived time for self-study as most needed to improve 
their level of competence. Almost half felt competent to 
teach about the computer and to teach using the 
computer. 

Home economics educators indicated a lower level of 
computer competence than other vocational educators. 
They also indicated a lower level of competence to teach 
about and teach using computers. 

Respondents in the total group indicated that vocational 
students frequently used the computer for word 
processing. Students used the computer less often for 
database, spreadsheets, and tutorials. Students rarely use 
the computer for programming and CAD. The 
respondents indicated that class assignments and projects 
would increase computer competence. 

Respondents self-rated level of computer competence is 
low if vocational educators should be on the cutting 
edge of technology. This level of competence may be 



the result of self-study as the primary method of 
computer training. Most respondents indicated that time 
for self-study would provide the most help in improving 
their level of competence. Few recognized the value of 
formal university courses or technical workshops as a 
method to learn the intricacies of working with 
computer hardware and software. 

The respondents make use of word processing for day to 
day writing tasks. However, they do not take advantage 
of other technology impropriate for use in their areas. 

Respondents indicated that students frequently use the 
computer for word processing and other tasks. The 
respondents also indicated that class assignments and 
projects using the computer would improve student's 
level of computer competence. However, it is possible 
that the level of competence held by the teacher 
educators would not allow them to plan, assign, and 
teach about projects suitable for expanding students* 
computer competence. Home economics educators also 
indicated a need for more computer hardware and 
software to improve students* competence. 

Introduction 

Computers are an important part of education in toda/s 
high tech world. The first national assessment of 
computer competency revealed that 87 percent of 
students have used a computer by the eleventh grade 
(Martinez and Mead, 1988). Tweeten (1988) stated that 
students entering school today already possess a degree 
of computer literacy.. Often, they arc more 
knowledgeable about computers than their teachers. 
Students' progress toward computer literacy can be 
hindered by their teacher's lack of knowledge. 

Although most schools acquired computers, many 
teachers found themselves inadequately prepared to teach 
about computers. Most graduated from college before 
many computer courses were offered. Many teachers do 
not feel confident about their computer knowledge and 
skitls(Martinez and Mead, 1988) 

It is an obligation of teacher educators to inu-oduce 
computer technology to both new and experienced 
teachers"The ultimate success of the microcomputer in 
education depends on it becoming an everyday 
classroom tool, much as chalk and pencils are, rather 
than being an object of special attention." (Lockard. 
Abrams, and Many, 1987, p. vii) To accept the 
computer as an everyday tool, teachers must be 
comfortable using it for personal and educational tasks, 
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and they muet be committed to making effective use of 
computers. (Lockard, Abrams, and Many, 1987) 

Teacher educators must assess their own skills and 
attitudes about computers because they are responsible 
for teaching computer competencies. If teacher educators 
lack the skills necessary to teach about computer use, 
they must seek to change their attitudes and level of 
competency. 

Educational change and the implementation of new or 
different systems are difficult at best. The situation is 
made even more difficult when a new or unfamiliar 
technology is involved. The change will be somewhat 
threatening to a few individuals. Therefore, an 
understanding of the change process and methods for 
planning and implementing change is most important 
This insight, coupled with sufficient knowledge of 
computer-based education, can bring about the 'critical 
mass* necessary for success. (Bozeman, 1985, p. 192) 

Purpose of the Study 

The purpose of this study was to compare home 
economics teacher educators self-rated level of computer 
com]>etence to those of agriculture, industrial, and 
business teacher educates. The study also determined 
vocational students* use of computers and suggesHons 
of what was needed to improve computer competence. 

Methods 

A random sample of 19 1 home economics, agriculture, 
industrial, and business teacher educat(»rs was selected 
from professional directories for participation in the 
study. One teacher educator in each vocational area was 
selected from all states with vocational programs. 

A questionnaire 6e\doped by the researchers was used to 
gather data. The questionnaire items examined the areas 
of use of computer hardware and software, level of 
competence, what is needed to improve personal 
computer competence, use of computers by vocational 
students, and what is needed to improve student's 
competence. The demographic variables included age, 
years of experience, gender, computer training, and type 
of computer used 

A total of 109 questionnaires were returned from 20 
home economics educators, 26 industrial educators, 39 
agriculture educators, and 24 business educators. 

The majority (55.56%) of home economics educators 
responding to the questionnaire were 41-50 years of age 
while 35.51% of Uie total group were 41-50 years of 
age and 32.7% were 51-60 years of age. Approximately 
one-thirds of the home economics respondents reported 
0-10 years experience (33.33%>, 1 1-20 years experience 
(27.78%), and 21-30 years experience (33.22%). In the 
total group, 25.23% reported 0-10 years experience, 
36.45% reported 11-20 years experience, and 26.17% 
reported 21-30 years experience. Most (88.89%) of the 



Teacher Perceptions of Computer Competencies 

home economics educators were female while 77.57% 
of the overall group was male. 

Most respondents in all groups reported that they 
received their computer training from self-study. Some 
reported taking university courses or inservice 
workshops. 

Table 1. Computer Training 





Self- 
Taught 


Univ. 
Course 


Inservice 


Dealer 
Course 


All 


72.90% 


9.35% 


11.21% 


1.87% 


HE 


44.44% 


33.33% 


5.56% 


0.00% 


IE 


80.77% 


3.85% 


7.69% 


0.00% 


AE 


74.36% 


5.13% 


15.38% 


5.13% 


BE 


83.33% 


4.17% 


12.50% 


0.00% 



Most vocational educators used an IBM computer 
(65.42%) followed by Macintosh (20.56%) and Apple 
II/GS (9.35%). Sixty-one percent of home economics 
respondents used IBM followed by 16.67% using a 
Macintosh, and 11.11% using an Apple II/GS 
computer. 

Findings 

When asked to rate their level of competence of 
computer use, the overall group rqxmed a mean of 2.63 
which indicated a level between good (3) and fair (2). 

Figure 1. Computer Competence of Vocational 
Educators 



Very 4 
Good 



Goods . 



Fair 2- 



Poon _ 
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Home economics educators rated their mean level of 
competence at 2.39, Industrial educators rated a mean 
level of competence of 2.58, agriculture educators rated 
a mean level of competence of 2.44, and business 
educators rated a mean level of competence of 2.%. All 
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individual gioups rated their level of competence 
between good and fiair. 

Most respondents in the total group reported using the 
computer (67^9%) and printer (65.42%) almost 
everyday, while never using a modem (54.21%) or 
scanner (73.83%). Sixty-one percent of die it^KMKlents 
in the home economics group repotted using a computer 
almost everyday and 55.56% reported using a printer 
almost everyday. In the home economics group, 
72.22% never used a modem and 88.89% never used a 
scanner. 

Word processing was used by 6729% of all le^xxidents 
daily while 55.^% of home economics educators used 
word processing daily. Other computer i^licadons used 
by specific vocational groins reflected the needs of their 
ateas. 

Approximately 22% of home economics rei^dents 
used databases daily or weekly while 38.89% used them 
once a month and 38.89% never used them. Only 
11.11% of the home economics respondents used 
spreadsheets once a week, 38.89% used ^readsheets 
once a month, and 50% never used spread^ieets. Most 
respondents (88.89%) never used CAD (Cbmputcr Aided 
Design). 

Industrial education respondents reported that 61.54% 
never used a database and 53.85% never used 
spreadsheets. Daily use of CAD was reported by 
23 .08 % of the respondents, weekly use was report by 
7.69% of the respondents, mondily use was rq)orted by 
19.23%, and 50% rqx)rted never using CAD. 

Agriculture respondents reported that 35.9% used 
database qjpUcations daUy or weekly and 51.28% used 
spreadsheets daily or weekly. Approximately 95% 
rqpoited that CAD was never used 

Business req)ondents rq)orted that 68.18% use database 
application daily or weekly and 66.67% use q)readsheet 
applfcations daily or weekly, .^yproximately 79% rf the 
business respondents teponeA that they never used 
CAD. 

When asked to identify what would be most helpful in 
improving their level of competence, 49.53% of the 
respondents indicated time for self*study, 31.78% 
indicated technical woricshops, 13.08% indicated 
availabili^ of funds for hardware and software and 
0.93% indicated university courses. Approximately 61% 
of the home economics respondents indicated time for 
self-study as most helpful in improving level of 
competence. 

Most (75.7%) rqported that they did not teach computer 
courses. However, approximately half of the 
respondents felt adequately prepared to teach about 
computers (45.79%) and teach using computers 
(52.34%). The perct;ntage was lower for home 
economics educators. (Rgures 2 and 3) 



Figure 2: C ompetence to Teach About Computers 
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When asked to indicate competence to teach computer 
applications, the percentage of home economics 
educators responding yes was tower tfian the total group 
in all areas (Rgure 4). 

Re^ndents were also asked to indkated student use of 
computers. Almost 85% of the respondents indicated 
that students in their area used a computer daily or once 
a week for word processing. Other q^lkations were 
used less firequendy by aU groups (Table 2). 

Figure 3: C ompetence to Teach Using Computers 
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Almost half of aU reqxMidents (50.47%) and 3333% of 
home economics respondents indicated'ttiat projects and 
class assignments would be most helpful in improving 
students* level of computer competence. Most home 
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economics educators (44.44%) ranked availability of 
hardware and software as most helj^uL 

Figure 4. Competence to Teach About Computer 
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Summary 

All vocational groups showed similar levels of 
computer compeience and use. Most ranked diemsdves 
fair to good oo levels of computer competence. Most 
used the computer and printer for word processing. 
Almost all vocational educators were self-taught and 
petcdved time for self-study as inost needed to improve 
their level of coiiq)etence. Almost half felt compe^t to 
teach about the computer and to teach using the 
computer. 

Home economics educat(MfS indicated a lower level of 
computer competence than other vocational educators. 



Teacher Percq>tions of Computer Competencies 

They also indicated a lower level competence to teach 
about and teach using computers^ 

Respondents in the total group indicated that vocational 
students frequently used the computer for word 
processing. Students used the computer less oflfen for 
database, q)readsheets« and tutorials. Students rarely use 
the computer for programming and CAD. The 
respondents indicated that class assignments and projects 
woukl mcrease computer competence. 

Conclusion 

Respondents self-rated level of computer competence is 
low if vocational educators should be on the cutting 
edge of technology. This level of competence may be 
the result of self-study as the primary method of 
computer training. Most reqxMidents indicated that time 
for self-study would provide the most he^ in improving 
their level of compe^nce. Few recognized the value of 
formal univer^ty courses or technical worksh(q>s as a 
method to learn the intricacies of working with 
computer hardware and software. 

The respondents make use of word processing for day to 
day writing tasks. Howevert they do not take advantage 
ofolberteduK)k3gy^ipropriateforusein their areas. 

Respondents indicated that students frequently use the 
computer for word processing and other tasks. The 
respondents also indicated that class assignments and 
projects using the computer would improve student's 
level of computer competence. Ifowevert it is possible 
that the level of competence held by the teacher 
educators would not allow them to plan, assign, and 
teach about projects suitable for expanding students* 
computer competence. Home economics educators also 
indicated a need for more computer hardware and 
software to improve students* competence. 



Tabic 1. Snidents' Use of Cbmputers 





Almost ] 


Everyday 


Once a Week 


Once a Month 


Never 


AU 


HE 


AU 


HE 


All 


HE 


AU 


HE 


Word Process 


47.66% 


44.44% 


37.38% 


33.33% 


6.54% 


11.11% 


3.74% 


536% 


Database 


10.28% 


536% 


27.10% 


22.22% 


27.10% 


22.22% 


28.04% 


44.44% 


SDreadsheeta 


10.28% 


0.00% 


29.91% 


16.67% 


27.10% 


22.22% 


27.10% 


5536% 


Proeranninc 


3.74% 


0.00% 


4.67% 


5.56% 


24.30% 


22.22% 


57.94% 


66.67% 


Tutorials 


634% 


0.00% 


28.97% 


27.78% 


33.64% 


27.78% 


22.43% 


38.89% 


CAD 


9J5% 


0.00% 


9J5% 


5.56% 


8.41% 


5.56% 


62.62% 


83J3% 


Statistics 


5.61% 


536% 


23.86% 


16.67% 


31.78% 


33.33% 


31.78% 


38.89% 


Testint 


4.67% 


536% 


18.69% 


22.22% 


34.58% 


22.22% 


33.64% 


44.44% 
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PROGRAMMING TECHNIQUES FOR INTERACTIVE 
CONCEPT DEVELOPMENT IN HYPERCARD 



James Baggotu Hahnemann University School of Medicine: and Sharon E. Dennis. Medical College of Pennsylvania 



Abstract 

Successful CAI for medical students needs to be 
infomalion-densc, yet clear and visually appealing. The 
show and hide commands of HyperCard can be applied 
in a user-controlled manner to reversibly build 
complicated narrative, tabular or graphic material. 
Techniques for each have been developed and are 
described 

Introduction 

Interactivity and user control are among the most valued 
principles of modem computer assisted instruction 
(CAI) design. Indeed, the attractiveness of hypermedia 
programs derives not only from the rich sensory 
experience of color graphics, animation, sound, etc., but 
from the potential for users to explore information in 
the way they wish, when they wish and as often or as 
little as they wish. CAI for pre-clinical medical students 
must, however, acknowledge the unusual constraints 
placed upon this category of adult learners by the 
crowded, lock-step medical school curriculum. These 
students are expected to master more information in less 
lime than probably any others. As a result, they are 
inclined toward learning techniques that they perceive as 
most likely to produce satisfactory performance in 
minimal time (Newble and Entwislle, 1986) 

The medical school environment thus challenges the 
CAI developer to facilitate learning large amounts of 
material efficiently and in a manner that will be 
interesting. As partial compensation, medical school 
offers a select group of learners, highly motivated 
individuals with above-average powers of concentration 
and ability to think abstractly. 

A dilemma imposed upon the medical CAI designer is 
that each display of information should be dense enough 
to be meaningful, yet it must avoid a cluttered 
appearance that would generate confusion. Information- 
dense screen displays can assist integration of 
knowledge by bringing out relationships among 
elements of information via a comprehensive visual 
impression. Further, highly motivated learners 
sometimes lose patience with programs whose 
intellectual reward for the effort of going from screen to 
screen is too small. 

Methods 

In writing Medical Biochemistry CAI intended to 
replace completely the corresponding traditional 
lectures, we have felt constrained not only to cover fully 
the lecture content, but also to present it at least as 
efficiently as a live lecturer would. This sometimes 
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requires trying to devise small-screen analogies to 
lecture techniques like explaining a large, complicated 
diagram or table that integrates a great deal of 
information. A lecturer would focus audience attention 
by pointing with a laser pointer to the pertinent regions 
of the projected image, would verbally develop each 
concept at length, and would relate each to other 
elements or to the overall picture as needed. The image 
would always remain before the audience as an 
organizational framework. It seemed important to 
mimic the essence of this style in CAI, and it seemed 
inappropriate to rely on the innate strengths of CAI, 
such as repeatability, multipath linking and problem 
solving in the hope of compensating for some less 
comprehensible and less efficient style of primary 
presentation. 

We have considered three kinds of information: 1) 
narrative (text), 2) tables and 3) pictorial material such 
as sequences of metabolic intermediates in a pathway or 
diagrams schematizing mechanisms of processes. In all 
cases, our CAI development has attempted to reflect the 
lecturer's style of organizing material, rather than 
attempting to force the lecturer into a new mold or to 
develop concepts from an altogether different 
perspective. This seems important, not only to assure 
the cooperation of the highly autonomous faculty, but 
also to avoid frustrating the already overworked 
students, who expect to be held responsible for 
everything in the lectures, and who might become 
confused by a style unlike what they had seen 
previously in lecture. (Many students choose to use 
CAI in addition to lecture, rather than as a lecture 
replacement) Predictably, CAI development sometimes 
leads to visual aids more effective than those previously 
available to the lecturers, and these may find their A'ay 
into the live presentations. 

Our way of incorporating interactivity and user control, 
while dealing with the information density/cluttered 
screen issue, is to use the show and hide commands of 
HyperCard. Major points of information are presented in 
bold typeface, and serve as advance organizers for the 
details, which arc typically in pop-up fields revealed by 
clicking on the boldfaced information. Any object 
revealed by clicking can be hidden by clicking on iL A 
check mark appears to indicate that the object has been 
viewed, and the check remains if the user hides it away. 

Narrative. / typical example of narrative development 
is shown in Figure 1. Panel A shows a screen prior to 
revealing details under the boldfaced major points, and 
Panel B shows the same screen after the details have 
been shown by clicking. Note the check marks. The 



240 



ability to hide and show flelds allows students to test 
their memories with mental drill. 

The rectangles on the right of the screen are buttons 
which give the student access to the Jcaturc named, such 
as a textbook reference, behavioral objective, an 
overview saeen listing all topics, etc. 

Figure 2 illustrates a text node. It consists of topics in 
underlined boldface type. Clicking on any of <hese items 
reveals more information on another screen. Any of this 
information may be further developed on subsequent 
screens. 

In all of these examples, the boldfaced material is 
contained in locked fields which are programmed to 
cause the appropriate event, such as showing a field 
containing details in plain text, when the mouse button 
is clicked while the cursor is within their boundaries. 
The fields containing plain text are programmed to hide 
themselves if clicked upon. Curiously, use of field 
programming in this manner has been ignored in ix)me 
books on HyperCard, but it makes a natural user 
interface with text 

Flelds can contain only one font in one size and style, 
but Greek letters, subscripts and related effects can be 
produced by layering scpmit transparent fields over one 
another. It then becomes necessary for the programmer 
to assure that clicking on any one of such a set of fields 
will produce the desired effect The user will expect that 
cliddng anywhere in a homogeneous-looking body of 
infomiation will have the same effect, and will be vexed 
if chance clicking on a small field containing a 
subscript fails to do the same thing as clicking on the 
rest of the text 

Tables. Good design of tables requires no more than 
four items at any level of subdivision (Willey and 
Jarecky, 1976). Within these constraints it is 
nevertheless possible to nutke large, complicated tables, 
which leave little or no screen space for explaining their 
contents. One way of dealing with this problem is to 
reveal the table piece by piece, using the blank space 
available before the table is fully shown for explanation 
of the information already showing. Figure 3 A 
illustrates a table as it appears initially. The segment 
which shows is explained in boldface, inviting the user 
to click on it. Figure 3 B shows the same table after 
two rounds of clicking to reveal successively more 
information. Note the two corresponding check marks. 
Clicking on the newly revealed information would hide 
it, and clicking on the new boldfaced type would show 
additional material. 

In this example the table is a HyperCard graphic, drawn 
entirely with paint tools, rather than text in fields. The 
boldfaced text is in overlaying opaque fields 
prognsmnmed to hide themselves upon clicking, thereby 
revealing the underlying table segment Simultaneously 
a transparent button over the newly revealed portion of 
the table is shown. The button is programmed to hide 
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itself and to show the corresponding boldfaced field. 
Proper layering of objects is critical for making this 
sequence operate correctly. 

Another approach to developing tabular information is 
shown in Figure 4. Here points requiring explanation 
are in boldfaced fields programmed so that clicking on 
them puts details into the rectangle at the bouom of the 
table. Clicking on other boldfaced fields replaces the 
contents of the rectangle with other details, and clicking 
on the rectangle empties it altogether. Notice that one 
cell of this table contains boldface imderlined material. 
Qicking here expands this cell, giving the student more 
information on ar<other screen. 

Choice of the technique to use depends in part on the 
importance of sequencing the information. In the table 
of Figure 4, the sequence is not critical. The 
conventions of the English language (left to right, top 
to bottom) are adequate cues; further, exploring this 
table in a **non-ided** sequence would be unlikely to 
cause confusion. In the table of Figure 3, en the other 
hand, the desired sequence of study is forced by the 
programming. 

Pictorial Material. Pictorial material resembles 
tabular information in that the problem is to build a 
detailed story without losing the overview; again, 
correct sequencing of information presentation may be 
important 

Figure S A-C illustrates one solution to this problem. 
Panel A shows the screen as it first appears; only the 
starling point, glucose, is visible. Subsequently the 
elements in panel B are revealed in the sequence 
indicated by the direction of the arrows, so the pathway 
appears to grow before the user's eyes. The effect is 
accomplished by placing empty, opaque fields over the 
information that is to be shown sequentially, then 
hiding these opaque fields with the screen locked, and 
finally unlocking the screen with a "wipe left slowly" 
or a "wipe right slowly** command, depending on the 
direction in which progression is desired. In keeping 
with our convention, clicking on boldfaced material 
reveals details, as shown in panel C. Since the user 
shows these details actively, there is no possibility of 
becoming confused as to which detail goes with which 
main point, as would be the case if panel C were the 
first thing the user saw. Information that should be 
merely a reminder of facts already known is placed in 
fields revealed with a "mouse within" command. For 
example, students are expected to recognize that PK 
abbreviates "pyruvate kinase," but a reminder is 
available by touching PK with the cursor (Figure S C). 
As soon as the cursor is moved away, the reminder 
disappears, avoiding unnecessary screen clutter. 

A second approach is shown in Figure 6. Panel A 
shows a typical metabolic pathway with multiple 
interrelationships. Only one point is in bold type; 
clicking on it changes the screen appearance to what is 
shown in panel B, with a narrative development of the 



231 



Baggou and Dennis 



point as described above. Clicking on filvfopi^n 
synthesis U iir#iv#- takes the user to another screen 
resembling Figure 6 B, except that now "Glycogen" 
would be in bold type. The user thus ^»egins learning 
about each element of the diagram with the entire 
picture in view. The narrative material, of course, is 
always placed on the screen in a position that does not 
obscure the point under development Consideration of 
Acetyl CoA, for example, which appears on the right- 
hand side of the screen, would be done cn the left half of 
thesaeen. 

Multistep transformations are not easy to describe 
effectively with static diagrams. The bei>t a lecturer can 
do without invoking animation (which is not usually 
available to the average lecturer) is to show a sequence 
of slides, each depicting a separate stage. This tends to 
lose the broad view of the process. Computer aninuttion 
solves the problem ideally, and in a way that cannot be 
approached with most lecture hall media. 

Figure 7 A is a schematic diagram of a cell in which a 
process is about to be started by the binding of a 
hormone (agonist) to a receptor on the cell membrane 
surface. Figure 7 B shows the final stale of the cell. The 
computer animates the process, so the user can see the 
agonist bind, the formation of intracellular 
intermediates, the migration of these intermediates to 
specific subcellular locations, and their immediate and 
final effects. Note, for example, how Ca++ has been 
released from storage, has bound to calmodulin, and 
how the Ca+-»^aI"^od"^in complex has interacted with 
and activated the protein kinase. The student initiates 
this sequence on the screen, and can lepeat it at will, 
ome of our students claim that it was not until they had 



seen the animation that they understood what was 
happening. 

Programming a sequence like this involves showing and 
hiding fields with the screen locked, and then unlocking 
the screen with a "wipe left/right slowly" command, as 
was the case for the example of Figure S. In addition, 
however, it is necessary to select and drag graphics in 
order to give the appearance of objects moving from 
place to place. To restore the screen so the student can 
repeat the sequence, a copy of the original state of the 
screen graphic, stored on a card that is never shown, is 
copied and pasted onto the screen, and the appropriate 
fields are hidden or shown. 

Conclusion 

Use of "show** and *'hide" commands in HyperCard 
permits CAI developers to design programs that are not 
only interactive, but that allow the students to build a 
great deal of information into a single visual 
impression. Our experience with first year medical and 
graduate students in a Medical Biochemistry course 
shows that the students have no difticulty with this 
inttiface. Once they grasp the principles of clicking on 
bold faced information to reveal more information, and 
of clicking on newly revealed material to hide it, they 
have no difficuHv in handling variations on these 
themes that may be tccessary to achieve special effects. 
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Hemoglobin, Gas Transport tc Allosterism 



Two closely related mammalian hemo- 
proteins bind oxygen reversibly, 

/ • Mb is found in muscle, is NOT on 
oxygen carrier 

— one polypeptide chain 

— one heme 

— binds one oxygen 

/ • Hb is found in the erythrocyte — 
carries oxyjien 

— Four polypeptide chains 

~ One heme per chain for a total of four 
hemes 

— Binds one oxygen per heme for a total 
of four 



R«fttr«nc« 



Objective 



Strueturs 



structure 




Figure IB. Dettils revealed; note checkmarks, which remain if the user hides the details. 
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/ • Hb is found in the erythrocyte — 
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Figure IB. Dettils revealed; note checkmarks, which remain if the user hides the details. 
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Hemoglobin, Gas Transport <c Allosterism 


The binding of heme to globin affects 
the properties of heme in important 
ways. 

The structure of heme 




Obj*etiu« 




Heme in nemoqiODin or myogiODm 

Oxidation of Iron (II) to Iron (III) 
Is orevented bif the bfndlnq of 
heme to qlobln. 

The 'distal' histidine decreases 


Ob isct i vtt 




Object iv« 


stability of C=0 comolexes. 






Quettiona 
CUnProb 



1 IBLS Menu | 







Figure 2A . Text node. Each undeilined topic is developed on a different scteen« 
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TCA Cycle and Carbohydrate Metabolic Capacities of Various Tissues 



Process 


Type of Tissue 


Aerobic 


Anaerobic 


Liver 


Adipose 


Muscle 


Brain 


Erythrocyte 


TCA cycle 


+++ 


■?•++ 


+++ 


+++ 





All aerobic tissues have the tricarboxylic acid cycle. 



Previous Card 

Figure 3A. Obligatory sequential development of tabular information. Initial screen appearance. 
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TCA Cycle and Carbohydrate Metabolic Ceoacities of Various Tissues 



Process 


Type of Tissue 


Aerobic 


Anaerobic 


Liver 


Adipose 


Muscle 


Brain 


Erythrocyte 


TCA cycle 


+++ 


+++ 


+++ 


+++ 




Carbo- 
hydrate 

metab- 
olism 


Glycolysis 


+ 


+++ 


+++ 


+++ 


+++ 


Lactate 
production 

glucose -> lactate 


+ 


+ 


(+++ in 
exercise) 




+++ 


Lactate production, however, is maximal only in 
anaerobic tissues and in muscle, where the rate of 
glycolysis can outstrip the rate of aerobic metabolism 
(during exercise). 



Previous Card 

Figure 3B. Obligatory sequential development after some, but not all, information has been revealed. 
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Summary of Major Metabolic Hormone Actions 



Hormone 


Site of Action 


Mechanism 


Action 


Adrenaline 
(Epinephrine) 


Muscle, adipose 

L 1 YCi 


CAMP t 
Ca ** T 


Glycogenolysis T 


Glucagon 


Liver 


In most 
tissues. 
cAMP T . 

Changing the rate 
of protein synth. 


uiycogenoiysis | 
Gluconeogenesis t 
a.a. permeabll. f 
Glycolysis i 


ArlinnQP 

rn VJ 1 ^ woe 


LiDolusis t 


Insulin 


Musc'e, 

adipose, 

liver 


CAMP 
cGMP 

tyr kinase activ. 
prot. synth. 


Glycogen synth. T 
Fatty acid synth. T 
Glucose perm. T 
Lipolysis t 



<3 

MAIN TOPIC 



Figure 4A table with space reserved at the bottom for commentary on it"! contents. Clicking on bold faced items reveals 
appropriate ccaunents in the reserved space. 
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Concq>t Devdopment in HypeiCaid 





Blood 


Ai^ocrobit Cell \^ 






— Glucose 


1 

Glycolysis in the anaerobic cell: / 


1 


Significance: / 




Control points: ' 







Over viey- 1 






bistnietibns 






Take Notes 






Calculate 






rind 






VhatlSaiT 1 






(juestioim | 






Tflible* ] 






IBLSlifenu 1 






MAW TOPIC 



Figure 5A. DevelqHPent of pictmial material. Initial screen ^pcarance. 
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Figure SB. Development of pictorial material. After the elements of the pathway have been revealed in sequence. 
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ConcqH Developineitt in HypeiCazd 



Cell 



This ratio is the major 
control of anaerobic 
glycolysis. It affects PFK. 

G-6~P 



ATP 



AMP 



V Transport 
V Glucose — Glucose 



PK = Pyruvate 
kinase 



May also come 
from glycogen 
Glyceraldehyde-3-P if glycogen is 

present in the 



Stimulated by 
anaerobiosts 



Pyruvate 




cell. 



NADH 



► — Lactate/-^ Lactate 
J Must diffuse out of the 



Glycolysis in the t '^aerobic cell: ^^^^ ultimate disposal, 
y Significance: Only energy source in anaerobiosis. 

/ Control points: Touch each enzyme abbreviation 

with the cursor for details. 





^ Overyiev 






Instnictions 






TokeNotesj 






Calculate 






Find 






VhatlSavj 






Questions 






. Tables 






IBLSMenu 






<p 

MAIN TOPIC 



Figure 5C* Development of pictorial material. After further details have been revealed. Note the checkmarks, which 
remain if the user hides the details. 
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ConcqM Development in HypetCaid 



B1 ood 



Triglyceride ^ 
(VLDL) 



/ Glucose 



Glycogen 

f 



cAMPi 



Triglyceride 

[MaTonyl CoA t 



Glucose— ►6-6- P-^j^p 



Fatty acid 



Elevated blood glucose (up to 20 mM) affects liver metabolism by two 
mechanisms. 

a. increasing saturation of glucokinase increases 
glucose phosphorylation. 



b. Elevated blood glucose produces an increased insulin/ 
glucagon ratio. 



c. Gl ycogen synthesis is active. 



<3 

R»<um<oT»x< 



Figure 6B. Conibination of nanalive and pictorial development Narrative which appears aSter clicking on Glucose. 
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Agonist 



The phospholipose C-catalyzed 
sequence of events is shown here. 



Membrant 



A : * 't ^^^Phosphodiesterase (Phospholipase C)**: • • . V 
/ 515- — i i— — 



PIP 
2 



Ca^'^ storage 



jCalmoduliru 



Protein kinase 
(inactivt) 

ATP + Dcphospho-Protcin 1 



Do it 



Figure 7A. An animated sequenos. Initial screen iq)pearance. 
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r 1 The phospholipQse C-calQlyzed 

I Agonist | sequence of events is shown he 



here. 



M»ml>ran» 



PIP2 



h o5phodiester<se\(PhosphoHp*s» C)":.- '.~ 

diglyctrid* ^C-Kin*$*^ Protein kin«s* C 
♦ * ' (active) 



C>**stor«q»IPT 

C c*++, 




ATP Dophospho 
Protoin 

ADP + Phospho 

'^ ^Tmodu^ Protoin 



Protoin kinas* 
(aotiv*) 

ATP + Dephospho-Protein > Phospho-Protein + ADP 



CEUULAR 
RESPONSE! 



CEaULAR 
RESPONSE! 



Figure 7B. Rnal tppemact of the sequence at the end of the animation. 
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Abstract 

This paper discusses rcpurpos;>ig an educational 
videodisc using three different hypermedia programs: 
Linkway for MS-DOS computers, HyperCard for 
Macintosh, and Tutor Tech for Apple// computers. 
Comparisons made among the three environments 
include the flexibility, structure, ease of authoring, and 
required design changes. 

Introduction 

The notion of using a machine to provide connections 
between several pieces of information originated with 
Vannevar Bush in 194SA small amount of work was 
done in this area in the 1960*s and 1970's, but it was 
not until recently that hypermedia and hypertext became 
"household" words. 

The most common meaning of hypertext is a database 
that has active cross-references and allows the read(^ to 
'jump' to other parts of the database as desiied" 
(Shneiderman & Kearsley, p. 3). Hypermedia, on the 
other hand, "goes beyond the concept of text" (Stevens 
& Stevens, p. 6). The information in a hypermedia 
database might be text, graphics, sound, or video. This 
infonnation can be linked together using a hypermedia 
software application program, and it can be accessed in 
associative, nonlinear ways. 

The recent popularity of hypermedia can be attributed to 
the increased access to computers and to the availability 
of hypermedia software such as HyperCard. These 
software programs make authoring easier and have "the 
potential to give back to teachers the control they lately 
have ceded to software programmers" (Trotter, p. 
35).Teachers can now provide a wealth of instructional 
materials that can adapt to individual student interests. 

Although there are many application programs now on 
the market for hypermedia development, this paper will 
address three that are of special interest to educators. 
These three (HyperCard for the Macintosh, Tutor Tech 
for the Apple // family, and LinkWay for MS-DOS 
machines) are all inexpensive and relatively easy to use. 
They offer tremendous potential for teacher-created 
instnictional programs. 

System Requirements 

HyperCard is distributed free with new Macintosh 
computers. It requires 1 megabyte of RAM and 2 floppy 
drives. It can be run on a Mac Plus, Mac SE, Mac II, or 
any of the other new models. For the comparison study, 
a Mac SE was used (with a 20 MB hard drive). The 
HyperCard program is required for development and 
delivery.. 



Tutor Tech is distributed by Techwarc Inc. of AltanK)ntc 
Springs, Florida. The cost is $195, which includes one 
teacher diskette and license to create up to SO student 
diskettes. Tutor Tech will run on an Apple //e, //c, or 
GS with at least 128K RAM. One floppy drive is 
required, although two are recommended. A hand control 
(such as a mouse, joystick, or koala pad) is required for 
the development, however the program can be delivered 
with either keyboard or hand control inputs. Delivery 
can be on a color or monochrome monitor. 

LinkWay is available from IBM and retails at 
{4)proximately $1 10 (A substantial discount is offered to 
educators.) Included in the price are additional programs 
for editing text, producing gnq)hic images, arid editing 
fonts. A free runtime delivery program is available upon 
request. LinkWay is able to run on almost any IBM PC 
or PS/2, or compatible computer. Although it can be 
created in modes up to VGA, the minimum 
requirements are 384K RAM and a CGA card A mouse 
is required for both development and delivery. See 
Figure 1 for a summary of the system requirements. 





HyperCard 


Tutor. Tech 


UnkWay 


Computer 


Macintosh 


Apple lie, 
IIcGS 


IBM 


RAM 


1 Mb 


128 k 


384 k 


Display 


Standards 


Mono 


CGA 


Input 


Mouse 


Hmd 
Control 
(Develop- 
ment) 


Mouse 



Similarities 

All three of the application programs contain the same 
basic structure, with slightly different terminology. 
Each screen in HyperCard and Tutor Tech is called a 
card, whereas in LinkWay they are referred to as pages. 
A collection of cards then becomes a stack (HyperC^d, 
Tutor Tech) or a folder (LinkWay). Each system also 
allows for the creation of objects on the screen, which 
are either buttons, flelds, or gr^hics. In addition, all of 
the programs can access and control videodisc players. 

Another similarity is the variety of levels of user 
access. Both HyperCard and LinkWay have five levels 
of control, ranging from total "freedom" for the author 
or developer to "read-only" for the student. The Tutor 
Tech program has 2 levels of control, determined by 
whether the Teacher Disk or Student Disk is the boot 
disk. All of the systems allow the author to easily 
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change from one level to another to "preview** the 
lesson from a student*s perspective. 

The creation and importation of graphics is an integral 
part of all of the systems. HyperCard has built-in 
paint tools similar to MacPaint. These tools are quite 
extensive with features such as fat-bits and stretching. 
Of course a wide variety of clip art may also be 
imported through the clipboard. 

The paint program in Tutor Tech is object oriented and 
more limited, with features of rectangles, circles and 
lines. However, there are many different colors and 
patterns available, and the graphics may be cut and 
pasted between cards or stacks. A sample disk is 
provided with the program that contains a variety of clip 
art, and it is easy to import graphics from programs 
such as Newsroom and Print Shop. 

Graphics are created in LinkWay in a supplemental 
program called LWPaint This program may be accessed 
from within LinkWay or from DOS. It is an "easy-to- 
use drawing application packed with standard features 
like pencil, line, box, circle, and text" (Pogue, p. 38). 
However, it does not provide many of the more 
sophisticated features such as the ability to stretch or 
shrink graphics. To provide more flexibility, LinkWay 
includes a program called LWCapture. This is a 
memory resident program that can be used to c^qpture 
any graphic screen of the same mode. LWCapture saves 
the image in a LinkWay format, which can be edited 
later with LWPaint^and called directly into LinkWay. 
All graphic Hies in LinkWay are stored as external 
picture files, which are read from disk when needed for 
display. Another similarity among the systems is the 
function of the buttons. In all three programs, the 
button is the primary object used to initiate action (such 
as branching to another card or calling a video 
sequence). In all cases, the button can be visible or 
invisible and can be almost any rectangular size and 
shape. 

DIffertnces 

A major difference between the programs is the script or 
language levels (see Figure 2). In HyperCard, the 
language is Hyper. Talk. It consists of an "English- 
like** language with 80 different commands and 
functions, and it is extremely powerful. Hyper. Talk can 
be used to create scripts for stacks, cards, buttons, 
Helds, or backgrounds. Animation is created with either 
a sequence of show cards or by choosing a menu tool 
from within a script HyperCard is the only program of 
the three with commands for visual effects (such as 
wipe and dissolve). 

There is a similar script language in LinkWay, which 
has about 40 built-in commands and functions"In 
certain ways it is similar to BASIC or Fortran** 
(Harrington, p. 283). In LinkWay, scripts can be 
written only for buttons, and they operate similar to the 
mouseUp ntessage in Hyper. Talk. 





HypciCanl 


Tutor. Tech 


UnkWay 


Commands 


80 


Embeixl 


40 


Animation 


Show Cards 


Wait 


Toolkit 


Fields 


Flexible 


None 


Restricted 


Grat)hics 


Bit Mao 


Object 


Bit Map 


User Access 


5 levels 


2 levels 


5 levels 



The openStack message in HyperTalk corresponds to 
the conditions for executing the LmkWay AUTOEXEC 
button script (a unique button that is activated when the 
folder is opened). Animation is possible in LinkWay 
with the purchase of an additional program called 
LinkWay Toolkit. Tutor Tech does not contain a script 
language; however, commands can be embedded on the 
ca''d that will activate when the card is c^ned. For 
example, the command •2« 12345SE 12355PL, will 
search for frame 12345 and play through frame 12355 
each time the card is opened. Since this command 
cannot be written as a script for a button, video can 
only be accessed when a card is opened. A similar 
embedded command can be used to produce animation as 
the result of a series of "wait** commands. 

Another major difference among the systems is the 
structure of the fields (a block of text that is treated as 
an object). Fields in HyperCard are quite flexible — 
they may be resized and easily repositioned. Fields in 
LinkWay are much more restricted, with no word wrap 
or automatic reformatting. Tutor Tech does not have 
fields per se. Instead, text is created using the text input 
for the object-oriented graphics. Each line of text is then 
addressed as a graphic. Fields in HyperCard can use one 
of many text sizes; in both LinkWay and Tutor Tech 
there are four sizes. The differences required for videodisc 
control are illustrated in Figure 3. n most cases, the 
HyperCard video drivers are purchased separately Erom a 
company such as Voyager or Apple. The example 
scripts on the chart reflect the Voyager commands. The 
Voyager stack contains drivers for several models of 
Sony, Pioneer, and Hitachi players. 

Figure 3> Video Acc ess 





HypeK:ard 


Tutor. Tech 


LinkWair 


Port 


Modem 


Serial/ 
Modem 


Serial 


Set Baud 


Voyager 


Dip 
Switches 


DOS Mode 


Initialize 


Video Init 


•2- SA 


1 line script 


Search 


Search, xx 


•2- xxSE 


Serial 1, 
"xxSE"; 


Play 


Play, till, 

XX 


•2- xxPL 


Serial 1, 
"xxPL"; 



Video in Tutor Tech is accessed via a serial card or (for a 
GS) the MODEM port. The embedded command •2- SA 
refers to the slot of the serial card (in this case slot 2). 
Tutor Tech can also access a videotape player via a BCD 
box. 
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LinkWay accesses the videodisc player via the serial 
poit The DOS mode command (Mode Bauds4800» in 
the case of DOS 4.0) can be used to set the baud rate. A 
button containing 9 lines of script is provided with 
Linkway to initialize the playerCThis button can be 
placed in the AUTOEXEC button of the opening 
folder.) Access of the video consists of a single line of 
code as iqMtsented in die chart 

Design 

To provide a realistic comparison, the three hypermedia 
programs were each used to repuxpose the Advanced 
Technologies Disc by the National Science Center for 
Communications and Electronics (NSCCE). This 
videodisc was produced by NSCCE for Level I delivery 
and contains die following five chapters. 

1. CD-ROM -Cdmpact Disc-Read Only Memory 

2. IVD-ImerMiveVkleodisc 

3. CDI- Compact Disc IntenK:tive 

4. DVI-Digittl Video Interactive 

5. Tdeoonfcftncing 

To provide consistency between the visuals on the video 
monitor and the computer monitor, the Main Menu of 
the videodisc was digitized, buttons were added, and it 
then became the Main Menu of the HyperCard stack 



(see Figure 4). The program is designed so that 
whenever the user returns to the computer Main Menu, 
a still frame of the Main Menu of the videodisc will 
appear on the video monitor. 

Utilizing a tutorial strategy, the resulting Leve in 
design features a submenu for each chq>ter with an 
Overview, COnfiguratioo, and ^iiriicatkms section (see 
Figure S). Sections can be accessed in any order, and 
they branch to a combination of motion video 
sequences, video still firames, and compu^ di^lays. 
Each segment concludes with a summary of the 
computer and one or two optional questions. Maximum 
learner control is provided with permanent options such 
as Review, Continue, Return to Menu and Exit In 
addition, video s^ments can be repealed. 

Hie **overview** sections furnish an introdoctkAi t. tach 
technology by providing general information and by 
highlighting unique features. A definition of terms is 
included to allow the learner to become familiar with 
the terminology associated with each technology. 
Motion video segments introduce each section and thim 
branch to grjphks and text screens on the computer. 




Figure 4, Main Menu 
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Fifure 5* Submenu for CDI aectkm 

The *'configi;iraUon*' sections for each technology 
provide a diagram of the necessary hardware. Each 
component pait is identified and described (see Figure 6 
for an IVD conn^umtion screen)* If the videodisc 
contains foouge for the configuration, it is also 
accessed 

The *'4)pUcations*' se:iions highlight several areas of 
use tor each technology. The learner may cbo^ to read 
the text screen which describes the ^)idications, or they 
may view them in ''action*' by accessing the examples 
provided on the videodisc* In the IVD and CD-ROM 
segments, the address and telephone number of several 
publishers are also provided for those seeking further 
infomuuion* 



□•v«lopm«nt 

It was relatively easy to maintain die majority of the 
design features throughout die three development 
efforts* Basic graphics and buttons were duplicated, and 
the branching remained quite consistent Because Tutcx^ 
Tech can only access video when a card opens, a few 
additional cards had to be created in that version. (In the 
other versions, video was sometimes accessed via a 
button, without branching to another card.) 

An advantage of HypexCard is that a video still can be 
called in the openCeurd command of a card script This 
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feature was used to return to a specific still frame 
whenever die learner letumed to die main menu, etc. In 
Linkway, only buttons can access video; dierefore, each 
button duu branched to die menu had to have die video 
script in it In die Tutor Tech version, it was easy to 
access die still frame when die card opened; however, 
some cards had to be di^licated because die first time 
die card was accessed was at die end of a motion 
sequence; subsequent limes itquired only a stiU frame. 

The picture pop-up and text pop-up features are unique 
to Linkway. Although diey can be simulated in 
HyperCard with scripts, they are not nearly as easy to 
incorporate. These features were added to die design in 
die Linkway version. There are no similar features, and 
no scripts to create diem, in Tbtor Tech. 

For remediation and feedback, die answer command 
(HypeiCard) and die message command (LinkWay) are 
almost identical They automatically provide a pop-up 
window on the screen with only one line of script. 
There is no corresponding feature in Tutor Tech. For 
diat version, die suident was branched to anodier card 
for remediation. Tutor Tech does have a built-in 
motivational card feature for die correct answer. This 
displays a special card for aperiod of 2 seconds before 
returning to die program. Tutor Tech also keeps score 
of the number of correct and incorrect answers and 
displays the final score when die student exits. 
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System F^aturM and Limitations 

From a developer*s point of view» the best features of 
HyperCard are iu thortcuu with keystrokes, the 
hypermap or recent option* the Hyper. Talk language* 
and the field flexibility. Especially useful to the 
computer*based author are the answer conunands and the 
hilite property. They allow the incorporation of 
feedback and the tracking of student progn^ss. The nuyor 
limitation of HypeiCard is tht limitation of a Mac SE 
— a snudl black and white screen. 

Linkway's attributes include its color (which can be 
CGA. MCGA» EGA, or VGA), the picture pop-up 
feature, and the external graphic files. By creating 
external files for the graphics and objects, it is very easy 
for the author to revise segments and cut and paste 
objects without going back into the original page. The 
limitations noted in LinkWay include the XOR colors, 
the lack of shortcuts for tiie pull-down menus, and the 
character cell alignment of fidds. 

The outstanding features of Tutor Tech are its ease of 
authoring and its flexibility of delivery. Because there is 
no underlying language, this system is extremely easy 
to learn and use. It serves as a good introductiOii to 
hypermedia, without the frustration of learning a 
language. The fact that Tutor Tech can be delivered with 



or without a mouse on an Apple //e, //c, or GS aUows 
for a large delivery audience. In addition, Tutor Tech is 
the only program of the three that has a built-in 
management system to keq> the students* score. The 
limitations of Tutor Tech include the ""bleeding** of tiie 
colors caused by tiie Apple // resolution, Uie 
predetermined sequence of button access for computers 
witixmt a mouse, and the fact that the size of each stack 
is limited to 32iC 

Conclu8lor< 

It is now possible for educators to create hypermedia 
programs whether they have an Applc//e wiUi 128K, a 
Macintosh, or an IBM witii only a CGA carl The tools 
are available which are botii flexible and inexpensive. 
The possibilities for changing tiie way teachers teach 
and students learn are limited only by our imaginations. 
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Abstract 

A modified hypermedia approach was used during the 
dcvelc^ment of an educational software program entitled 
ATLAS plus (Advanced Tools for Learning Anatomical 
Structure) at the University of Michigan Medical 
School. This approach allows the user both flexibility 
and structure within the same software program. The 
philosophy behind this approach, a brief description of 
the software program, specific examples of the design 
features created, and expectations of users' needs are 
discussed 

Introduction 

The use of hypermedia in educational settings has 
quickly captured the imagination of educators and 
instructional designers from elementary through higher 
education. Newly emerging technologies have allowed 
software developers to challenge the constraints of 
traditionally defined computer assisted instruction, 
allowing users the flexibility to navigate through 
content and functions in a manner unique to each 
individual (Frisse, 1990). In medical education, this has 
resulted in numerous projects designed to deliver large 
amounts of textual, graphic and image material in 
efficient and easily accessible formats. The opportunity 
to deliver content so efficiently has distinct advantages 
in a Held where the infonnation students must learn 
increases exponentially each year. Additionally, more 
and more medical schools across the country are shifting 
the focus of their curricula to reflect the need to teach 
students to be lifelong learners; that is provide more 
student directed, student generated learning activities. To 
this end medical educators arc currently using or 
develc^ing hypennedia programs to teach basic science 
material, such as anatomy (Zagari, 1989) and pathology 
(Tessler, 1990), as well as clinical problem solving, 
diagnostic techniques (Greenes, 1986) and decision 
making (Greenes, 1986). 

Students in the medical school environment are a 
relatively homogeneous population; they are generally 
high performing as well as highly motivated learners. 
There are, however, students who enter medical school 
who may not fit this stereotypical mold. The standard 
curriculum which requires that students sit in lectures 
and laboratories all day for two years before spending 
two years in clinical exjoricnccs may not be the most 
effective teaching method for reaching this group. 

The purpose of this paper is to describe an educational 
software program that was developed to meet the needs 
of all types of learners; essentially it established 



structure within a hypermedia type environment. This 
structure does not imply merely a linear approach to the 
content. Instead, the links and associations among 
screens, modules, and content have been anticipated by 
the content experts and designers. The software 
developers are, in a sense, predicting any additional 
information that students of varying sophistication 
might need and providing easy access to that content 
This guided approach to the content represents the 
foundation for the design philosophy underlying the 
software development A description of the project will 
be provided, with a focus on varioas design techniques 
for applying this philosophy to the creation of 
educational software. 

Software Description 

ATLAS plus. Advanced Tools for Learning Anatomical 
Structure, is a teaching tool reflecting a modified 
hypennedia approach to the study of the anatomical 
sciences. The software program integrates the use of 
digitized images, pictures, computer graphics, sounds, 
animations and textual infonnation to dynamically teach 
basic concepts and principles of several anatomy 
disciplines. ATLAS plus operates on a Macintosh Ilex 
ethemet network. 

Tutorials in histology (epithelium, connective tissue, 
bone structure and growth, muscle), embryology 
(development of the human face, teratology), and gross 
anatomy (respiratory system, digestive system, nervous 
system) have been developed to date. Additionally, 
several laboratory introductions to histology have been 
completed, as well as data banks of drill and review 
questions, and a comprehensive examination program. 

The primary authoring tool used was Authorware 
Professional by Authorware, Inc. Additionally, 
HyperCard (Apple Computer. Inc.), SuperCard (Silicon 
Beach Software), Modem Artist (Computer Friends), 
and Macromind Director (Macromind) were used to 
supplement the authoring system. This allowed a 
variety of functionalities while creating and maintaining 
a seamless environment for the user. 

Design Techniques 

A variety of design techniques was established to be 
used consistently across all modules of the software 
program. Again, all were devised to allow both 
flexibility as well as structure for the user. The 
following sections address several of these techniques in 
detail. 
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Menus 

A »nenu system was developed to facilitate and allow 
the user to take cither a structured path through the 
lessons or operate in a hypermedia fashion, jumping 
from component to component in any order. Each 
lesson is broken down into a number of components, 
each component accessible from a main lesson menu. 
Normally the lesson menu is designed so that a 
suggested path is apparent. For example, a menu might 
have component options listed in a top-down fashion 
implying that the user start with the top option and 
move through each module in order. This is to insure 
that all components are covered in the order desired by 
the instructor. Or a menu might present component 
options in a circular or rectangular arrangement, again 
suggesting an order with a beginning and an ending 
clearly indicated. Often lesson components are broken 
down into sub components, each accessible from 
another sub menu, designed to cue the user to a 
structured path. Sub menus and sub components can run 
several levels deep, but all follow the same pattern of 
designating a lesson path. 

Dividing the lessons into components and further into 
sub CI iponents accomplishes two things: first, 
components arc kept to a reasonable length, and second, 
users can jump easily into specific areas of a lesson. 
Thus, because components arc accessible by menu, and 
because students can select any menu option available 



on a screen, they can easily use the menu options to get 
to a specific lesson component instead of tediously 
paging through a lesson. 

Directories 

Directories, like menus, provide the user with some 
degree of control over the navigational sequence through 
the material presented. At a very basic level, use of a 
directory can complement and improve the ngid page 
turning nature of a linear, screen by oi^reen presentation 
by allowing the learner to exit the linear presentation at 
any time and re-enter it at another spot of his or her 
choice. At more complex levels, additional options 
which build on the user's level of expertise and increase 
user control can be integrated into the directory. An 
example from ATLAS plus will be used to illustrate 
this point. 

One of the components of ATLAS plus is a scries of 
laboratory introductions. These lab introductions are 
designed to introduce students to the cells and tissues 
that they will be viewing under the microscope during 
their actual laboratory sessions. Digitized images are 
displayed along with accompanying textual explanauons 
which highlight the important structures or tissues. 
First time users of the lab introductions are encouraged 
to proceed through these lessons in a systematic, serial 
manner, starting with the first image and simply paging 
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through until all the images have been seen. However, 
in subsequent uses of the laboratory introductions, 
students may wish to simply review images, thereby 
requiring a different strategy - one that involves using a 
directory at increasingly sophisticated levels. 

At the most basic level, a directory within a given lab 
introduction consists of a series of titles corresponding 
to images available to the user. There is no sequence of 
viewing imposed on the user, aside from the fact that 
the listing of the images is arranged from the simple to 
the complex. Selection of a given title will result in the 
display of the corresponding image and a routing of the 
user back to a point in the serial pathway where the 
associated textual explanation is displayed. From this 
point, a user may continue on to the next image on the 
serial path, go back to the previous image on the serial 
path, or retiun to the directory to make another 
selection. This mode of the directory is aimed at those 
students who require a detailed description of the image 
displayed, yet desire more flexibility in selecting 
specific images than the serial pathway alone allows. 

Other dimensions of flexibility can be added to this 
basic directory to increase its level of functionality. For 
example, the designer may associate an option with 
each image title which, when invoked, simply displays 
the image without leaving the directory. This mode 
assumes that users are somewhat more familiar with the 
images in question and therefore need no accompanying 
textual explanations. It may be used effectively as a way 
for the user to informally self-assess his or her abilities 
to recognize certain images and to identify the important 
features. 

The third and most complex dimension builr into this 
directory is the ability to display and compare multiple 
images on a screen simultaneously. As each image is 
selected, it is displayed in a small window on the 
screen. Each of the small windows may be re-sized by 
the user. This mode assumes an even higher level of 
familiarity with the content, allowing the user to not 
only identify an image but also compare and contrast it 
to other similar images. Again, no explanatory text 
accompanies the images. 

Animations 

A number of the aninvition sequences in ATLAS plus 
are lengthy and describe complicated, multi-step 
processes. In these instances the animations are broken 
into several sections. The user may then choose to see 
each animation component in logical sequence or select 
an option to see the entire process animated in one 
continuous movie. The user may also play the 
animation components in any order to allow easy 
review of just one step in a given process, or opt to 
play a specific component several times in a row. 
Again, this design allows either a structured or freeform 
viewing of the material. 



Contextual Framework 

It is important for the user of a hypermedia application 
to understand his/her location in the lesson, both from a 
physical or navigational standpoint as well as from a 
content perspective. Various design techniques can be 
used to convey important contextual information. The 
purpose of this information is to define a set of 
expectations for the user which are reflective of the 
program's logical structure. It is the contextual 
framework then, that empowers the user while 
simultaneously helping to minimize confusion. 

To provide a navigational context, the use of a reference 
bar is helpful. A reference bar posts the total number of 
screens in the module and the specific number of current 
screen being viewed. Using this device, the user is 
informed of his relative location in the module. 
Assuming that each screen averages the same amount of 
information, the user can estimate the length of the 
module based on the total number of screens. The 
approximate amount of information remaining can be 
calculated at any point in the lesson. 

Information regarding content can also be relayed via the 
reference bar. For example, information in ATLAS plus 
is usuall) , ouped into lessons which are often further 
divided into modules and sub-modules. Each reference 
bar indicates the lesson name, module name, and sub- 
module name to inform the user of his location from a 
content perspective. The content information in a 
reference bar might read as follows: "Face: 'Setting the 
Stage:' The Embryonic Face: Weeks 4-8.' " This tells 
the user that within the the "Face" lesson, s/he is in the 
section "The Embryonic Face: Weeks 4-8," which is a 
sub-module of the ^'Setting the Stage" module. Thus, 
contextual information as it relates to the general 
content of the exercise is provided. Merging the sense of 
navigational context with contextual context, the user 
may begin to chart a course through the lesson. 

In ATLAS plus, a well-designed map of a given lesson 
can provide the user with 1) access to all choices at 
once, 2) the scope of content covered by each choice, 
and 3) a suggested order to follow (see Figure 1). This 
type of mapping structure, which provides mechanisms 
that both assist the user in determining "where" he is in 
a lesson, as well as informing him as to the options 
"ahead** of him, is an example of a design that promotes 
and allows effective and efficient database navigation. 
Navigational decisions that include content and/or time 
considerations on both a micro and macro level can be 
made. For example, limited by time, the user may 
decide to browse several large components covering 
familiar material, and then spend the bulk of his time 
on a unfamiliar, medium sized component. 

Informing the user of the the size and content of tne 
various components in a hypermedia database, as well 
as where he is in these components, and consequently, 
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where he is in the overall database improves his chances 
of using the database effectively. 

Links 

There are two types of links in a hypermedia letuning 
environment: unstructured and structured. In many 
ways, the power of hypermedia has been the promise of 
allowing the user to freely link different pieces of 
information and media, limited only by boundaries of 
the environment and the function of the program. When 
designing educational material, it is absolutely essential 
for the author to anticipate and create specific 
informational links. 

Unstructured Links. The ability to access any part 
of the lesson environment, either directly or indirectly 
via a series of menu selections, is an example of how 
the user can make links or associations in a free, 
unstructured manner. For example, a suident in the bone 
lesson might desire access to information in the 
connective tissue lesson. Because all lessons in a major 
subject area are accessible via pull down menus, the 
user may utilize a pull down menu to make this 
connective tissue link, resulting in an immediate 
routing to that lesson. Using the m^nu options in the 
lesson, the user then navigates to a specific point in the 
lesson. 

Another example of the ability to freely associate is in 
the use of the glossary. The glossary is always available 
to the user via a pull down menu. To Hnd a term in the 
glossary, one simply selects the glossary from a pull 
down menu and is linked directly to the glossary. In the 
glossary one can roam freely among the entries or look 
up specific words. In both of these instances, the lesson 
environment allows the user to jump freely from lesson 
to lesson, lesson to glossary, making connections 
between speciHc pieces of information that are not 
necessarily anticipated by the authors. 

Structured Links. As stated previously, it is 
important for the software developer to anticipate and 
create links. There are various types of links but with 
one common ability: they provide multiple and 
explicitly related connections among different pieces of 
infomnation. 

A link is often a doorway to a more detailed explanation 
of a subject: material review, greater depth, definitions 
and so on (see Figure 2). Other times, the link is 
merely another angle on the main source of 
information, or just a parallel source of information. 
Examples might include an illustration which 
accompanies an image, an inset, or a different image 
which provides another view of the same object. 
Sometimes the link is optional, as in the case of a hot 
button, giving the user an opportunity to explore a 
topic in greater depth; other times the link is automatic, 
as in the case of simultaneously presenting a digitized 
image on one screen and a simplified illustration of the 
image on tlie other. 



These structured links have three characteristics in 
common: first, the associations are specific; second, the 
associations arc anticipated by the author, and third, the 
associations, optional or automatic, are built into the 
lesson at a specific place and only available at those 
specific locations (unlike pull down links to other 
lessons and the glossary). 

The value of these structured links is that they are 
anticipated by the software developer who uses his or 
her expertise and experience to anticipate where users 
may have problems or may need more or different types 
of information on a topic. By anticipating and providing 
specific hyperlinks, a hypermedia environment is created 
that has elements of freeform hypermedia, but also 
provides a structure that has specific limits and teaching 
goals. Access to information is not left completely up 
to the whim or free association of the user. Thus, it is 
the responsibility of the developer of an educational 
hypermedia environment to provide a learning structure 
that gives guidance to the learning process; in other 
words, one which actually teaches rather than simply 
provides a free floating collection of related yet 
disjointed points of information. Anticipating and 
creating specific inf<Hmation links is one way guidance 
can be provided. 

Distillation of content Into succinct units 
of information 

In order to accommodate the various strategies that 
learners use to navigate through lessons, at the 
minimum, the basic conventions for moving forwara 
and then tracing the same path backward should be 
available at any given time. Traditional hypermedia 
environments have often placed the burden on the user 
to trace that "backward path" through the material in 
order to review a specific concept or point of 
information. However, lessons can be designed to 
extract content already covered and refer to it in such a 
way that the need for this review path is minimized. 
One advantage in using this technique, for the less 
sophisticated learner for example, is that it focuses 
attention on the salient points of the instruction, 
separating them from all accompanying detail.. 
Moreover, a context in which to present new 
information is established. 

The basic premise behind this technique is to provide 
users with frequent references to information previously 
displayed so that they may use it as a foundation for 
learning newer material. These references are composed 
of segments of instruction whicn have been distilled and 
assembled into succinct, discrete units of information. 
The units may contain text or graphics or a 
combination of both. They contain little of the detail 
that the initial learning point contained; rather, they 
focus on the general concept presented (see Figure 3). 
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Figure 4. 

Their puipose is to provide the learner with a sense of 
the prerequisite knowledge necessary to master the 
present concept, and to help transfer that knowledge to 
the present situation. These units also provide a 
navigational context because they always represent 
information that has previously been explained and 
clarified. Displaying these references directly on a 
portion of the user*s screen gives the user access to this 
information without having to go back and find the 
place where it was initially discussed. 

While use of this technique may obviate the need for the 
user to frequently *'go back and review,** imposing this 
structure does not prevent the user from tracing a 
"backward path** through the lesson if desired. In fact, 
users who need to go back and review the details and 
nuances of previously presented information should do 
so. At the same time, the technique enables those who 
are ready to move ahead to do so, taking with them a 
reminder and understanding of the knowledge required for 
furth^ learning. 

Quizzes 

Providing options from which the user may directly 
access quizzes is a common hypermedia design 
technique, lending itself well to building in the 
flexibility that many learners will likely use. Others 
may prefor to take quizzes in the context of the logical 
progression of instruction. Whether a quiz is chosen by 



the user or presented as a result of an assessment of the 
user's performance to date, several techniques can be 
used to structure the quiz experience. As quizzes are 
multipurpose by nature - they may be used as self- 
assessment tools, in preparation foe a formal evaluation, 
as teaching tools, or just f(v fun - the challenge to the 
designer is to anticipate these needs and integrate them 
into the quiz environment 

A random presentation of the questions decreases the 
likelihood that the quiz simply becomes an exercise in 
memorization. An option to see a cumulative score will 
satisfy those students who are curious without 
burdening those who would rather not know their 
scores. Options to restart the quiz at any time will 
beneflt those students who are using the quiz as a 
teaching ^ool or who are performing especially pooriy 
but now want to zs&tss themselves **for real.** A **quit at 
anytime** option is essential to those using the quiz in a 
teaching tool mode. 

But the key to desigiiing a multipurpose quiz lies in the 
provision of instructive, informational feedback, which 
the user may or may not choose to see. Good feedback 
not only tells the user whether the answer given is 
correct or incorrect, it provides teaching points (see 
Figure 4). Carefully constructed feedback may focus the 
learner on another dimension of the problem or suggest 
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a different strategy in approaching the problen); it 
should avoid revealing the answer. 

For example, a student wishing to use the quiz as a 
pretest to assess his or her weak areas may want to 
work through a representative number of questions and 
receive a meaningful score reflecting his or her grasp of 
the content matter. Another student who wants the 
benefit of the teaching information included in the 
feedback may go through the quiz and select all the 
incorrect answers to a given question simply to find out 
why they are incorrect. A combination of these two 
modes may be achieved if the quiz is structured in such 
a way that the score reflects only the user's initial 
response to the question. Hence, the learner would not 
be penalized for applying a hypermedia approach to 
seeking instruction, or, in other words, accessing the 
feedback to all possible responses. Instead, that 
flexibility is inherent in the design of the quiz. 



Examination Component 

Designing a computer-based examination component 
can be difficult in a hypermedia environment due to the 
rigidity often inherent in the nature of giving/taking 
exams. While it is important to carefully design the 
exam so that only the information specified by the 
exam author is accessible by the user, a good computer- 
based exam must offer the user the same navigational 
and contextual flexibility that a paper-based exam 
allows. That is, the user must be able to work through 
the exam in any order desired, skip and later return to 
specific questions, change and modify answers, place 
"tick marks" next to difficult questions for later review, 
make notes on a scratch pad if desired and review 
progress to date. A key element to making a user 
comfortable with taking a computer-based exam is to 
give the user full control when it comes to committing 
to one answer (x another. 



14. Which of the following cells become the macrophages of 
connective tissue upon leaving the blood stream? 



A. Monocycles 

B. Neutrophiiis 

C. Mast Cells 

D. Eoslnophllls 

E. Reticulocytes 
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An effective technique in designing an examination is 
the provision of a review or progress screen (sec Figure 
5). This screen displays the question numbers 
corresponding to each exam question along with the 
user's answers (or a b^ank. if the question has not yet 
been answered), and inaications regarding the status of a 
given question. For example, if a question has been 
"tick mariced" by the student for later review or if the 
student recorded some notes on the electronic scratch pad 
for that question, those actions are reflected on the 
review screen. The review screen is continuously 
updated throughout the exam so the student may assess 
his/her progress in an ongoing manner. Further, by 
linking each question number on the review screen to 
the corresponding question in the exam, the student may 
use the review screen to jump directly to any question 
in the exam. 

While it is important to provide an option for the user 
to exit the exam at will, it is also prudent to ensure that 
the user is satisfied that s/he has completed the exam in 
a thorough manner. To achieve this, a mechanism 
which automatically scans the student's progress can be 
invoked. This mechanism checks to see that all 
questions have indeed been answered and that no 
questions are still left "tick marked." If these two 
criteria are not satisfied, a message is displayed advising 
the user that an exit will be permitted only when all the 
questions are both answered and unmarked. The user is 
then automatically returned to the review screen to 
identify the specific questions that need to be re- visited. 

Finally* a computer-based multiple choice exam has the 
benefit of being able to tabulate a score immediately. 
An option to sec the examination score will satisfy the 
curious yet spare those who may not need/want to see 
that information right away.. 

Conclusion 

This paper has outlined various techniques for 
establishing structure within a hypermedia environment; 
that is. providing users various routes, both guided and 
unguidcd. through large bodies of electronic 
information. These techniques range from anticipating 
links for student paths through the program to 
providing directories of images and modules with 
suggested, but not mandated, paths indicated. This dual 



approach allows users to move freely from screen to 
screen or module to module charting their own courses, 
while also suggesting preferred pathways through the 
content. Thus, users at varying levels of sophistication 
and expertise can choose the learning modality most 
suited to their needs. Individuals new to the content may 
choose the suggested path, while others using the 
material for review or self-assessment may opt to 
approach the material in a more unstructured fashion. 
Therefore, unlike pure hypermedia environments, 
learners at all levels will be able to adapt the software to 
meet their own dynamic needs, exercising even more 
control over their own educational experiences. 

Finally, while this paper discusses a program designed 
specifically for use with medical students, the 
underlying philosophy is applicable to educational 
software programs for all levels and types of learners. 
By applying the techniques outlined above, educators 
will provide effective learning experiences for a greater 
range of students. Moreover, these students can become 
self-directed learners employing a range of problem 
solving and information acquisition skills in their quest 
for further knowledge and understanding. 
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Abstract 

"Teachers will never develop their own computer 
programs". HypeiCard has proven this su oient false. 
Data is presented on secondary teachers* success as 
novice computer program authors. Their work is 
compared to experienced authors* programs and is 
evaluated for |>otential users. The impact of the 
authoring proccs on the authors is assessed. The 
resulting Instructional Technology Guidebook is 
discussed. 

Introduction 

Several years ago, technology training planning 
committees in many school districts, decided not to 
even consider authoring their own software nor to teach 
the teachers how to do it**Teachers will never develop 
their own computer programs**, it was said**It*s too 
difficult and time consuming.** 

Today, HyperCard programs developed by Jefferson 
County Public School District (JCPS) high school 
teachers are available for use and as models for other 
Macintosh using teachers. While these programs may 
not be professional quality, they prove the point that 
teachers can develop useful CAT programs with a 
limited amount of training and development time. In 
addition, they demonstrate that in the authoring process 
teachers learn, not only about the operation of 
computers and software programs, but also more about 
their curriculum and how to teach it using technology. 

This paper addresses questions about what teachers can 
do given limited experience with an authoring tool, the 
usefulness of such learning to teachers, and the 
usefulness of the resulting stacks to others. 

CAI Authoring Tools 

The authoring process is one of the most challenging 
tasks in producing computer assisted instructional 
programs. It involves turning flowcharts, control 
algorithms, format sheets and other design 
documentation into computer code that will make the 
program operate correctly in the delivery system 
(Reeves, 1986). 

The alternative means for writing computer code are to 
use a programming language (BASIC, Pascal, C), an 
authoring language (PILOT, Coursewriter, TUTOR), or 
an authoring system (QUEST, TICCIT). A 
programming language is a set of commands for 
producing prescribed outcomes understood by both the 
programmer and the computer (Hannafm & Peck, 
1988). Although teachers can leam to use BASIC for 
relatively simple applications, a professional 



programmer is required for development of complex 
programs. 

An authoring system is a software package designed to 
lead the user through the process of creating CAI 
instruction without requiring programming (Hannafm & 
Peck, 1988). These systems are often restricted in 
flexibility although easier for the non-programmer 
teacher to use. Authoring languages offer the best 
compromise between the flexibility of programming 
languages and the ease of use of authoring systems. 
HypeiCard offers the advantages of both an authoring 
system and an authoring language. 

HypeiCard has been described as a CAI authoring tool, 
an information organizer, a four dimensional blank 
slate, a software erector set« and a hypermedia tool kit 
(Tsuk, 1989). The key to HyperCard authoring is 
HyperTalk, **a marvelously simple programming 
language** built into HyperCard (Sculley, 1988). With 
HyperTalk, virtually anyone can become a software 
author and produce professional looking Macintosh 
£q)plications. 

Three uses of HyperCard can be of value to teachers 
(Echols & Rosseu, 1989; AUen. Dodge & Saba, 1989). 
Use as a data base program allows linking of data 
elements in fields on various cards. As a hyper media 
vehicle, it presents and handles nonlinear text 
associations, graphics, music, speech, sound, 
animation, video disc and other external devices thereby 
eliminating the need for devices and products in some 
formats and coordinating the use of others. 

As an authoring tool, HyperCard has simple capabilities 
for the nonprogrammer to complex possibilities that 
challenge experienced programmers. Prototypes and 
storyboards can be prepared direcdy on computer to cut 
duplication of work in other formats. The authoring 
capabilities seem to have the greatest potential since 
Hyper. Talk is easy to leam and yields professional 
looking results. In addition, teachers master som*.* of the 
programming skills necessary to use digital 
technologies. 

HyperCard also has disadvantages which include limited 
audio capability, black/gray and white only, reduced 
fidelity and resolution in displays when compared to 
motion pictures and slides. But remedies are around the 
comer. 

Teachers as CAI Authors 

Teachers have expertise they can combine with a 
versatile authoring tool. They know the curriculum and 
are familiar with effective instructional approaches for 
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their «iudents. Also, many teachers long for more 
excili'ig and effective ways to present their instruction. 
With JCPS's establishment of Macintosh writing labs 
in hi^^h schools, all secondary teachers now have access 
to Hy|.<^4Card's capabilities. 

An obvious question is, why would teachers want to 
author HyperCard stacks? Those who have done it know 
that the authoring process can satisfy some deep 
motivational needs and desires as well as practical issues 
of software availability. Other reasons include these. 

1. ^'Designing and scripting your own stacks is 
neither so hard as to be intimidating nor so 
time consuming as to be prohibitive if 
programming isn't your primary occupation" 
(Shafcr 1988, pp. 2). 

2. If you want something done right, you have to 
do it yourself. Your stack can be the way you 
want it to be rather than someone else's idea. 

3. You understand better how to use something 
you've designed. 

4. The creative process itself is rewarding and 
enjoyable. 

5. In designing a stack to solve a problem, you 
will gain some insight into the nature of the 
problem itself and thus become a belter 
problem solver. 

6. Authoring gives a positive boost to your self 
esteem and fosters confidence in ability to 
master complex skills. 

7. It develops understanding and insight into the 
use and importance of computers and software 
in education (Heimler, Cunningham, Nevard, 
1987). 

Authoring stacks can also cause teachers to rethink how 
they present their curriculum to students. Computer 
based materials may be more meaningful and 
motivating. They can present information in a 
consistent manner. Teachers can more clo*::^<y monitor 
student learning. The teacher will discover a new role as 
facilitator of learning. 

HyperCard Program Development 

During 1989-90, 240 secondary teachers received 12 
hours of training on HyperCard as part of a larger 
inservice program on instructional use of Macintosh. 
Training was conducted for two groups of 120 each by 
the JCPS Computer Education Support (CES) teachers. 
The University of Louisville School of Education 
provided the opportunity to receive university credit for 
the "New Kid Graduates" training. It was hoped that 
many teachers would apply the hours to a 30 credit 
post-masters program with empha is on instructional 
technology. 

In order to qualify for university credit^ teachers had to 
complete two projects; one had to use HyperCard. The 
HyperCard project was intended to pull together 



HyperCard techniques, instructional design and content 
area curriculum. 

The 12 clock hour HyperCard training consisted of these 
topics: creating a simple stack with appropriate 
instructional and screen design; creating cards with 
background and with text in various size, font, styles; 
fields, buttons, simple scripting and linking; copying 
and creating graphics; scanned images; sound. On-site 
follow up support by CES teachers allowed those who 
were interested to receive further 1 on 1 assistance in 
completing their projects. 

From Phase I training (January to October), 30 
HyperCard stacks were completed. From Phase II (May 
to December), 70 were completed. The stacks were 
related to the high school curriculum or to use of the 
Macintosh. Topics ranged from "The Nine Planets" to 
"Nelworic Management", from "The Orient Expi«ss" to 
"HyperCard Sounds and Visual Effects Tutorial". Three 
of the stacks were for teacher use only, 66 for student 
use and 26 for use by both. Two were created for 
students by students under teacher supervision. Five 
were excluded for poor quality. About an equal number 
of stacks were developed for science, math and language 
as well as use of Macintosh and its programs. 

The stacks were analyzed to identify the components of 
HyperCard and authoring concepts used. The average 
stack contained 16 cards and ranged in size from 8 to 
135 cards. Most were intended to deliver information 
about a topic, were not interactive and did not assess 
learning. The next largest group was tutorials that 
provided information, examples, checked learning and 
included a post test. Some others provided drill and 
some were response shells that had students respond to 
questions about novels read and print the responses as a 
paper. One lAV driver stack was submitted. Some 
stacks were designed to be used concurrently with a data 
base, a spreadsheet or CD-ROM. Some had follow up 
assignments. 

Most stacks included a tide card, menu, introduction or 
overview. Few specifically stated the objective to be 
accomplished. Most had accurate and appropriately 
organized information. Many branched from the menu 
but some were linear. Some branched within the stack 
and some branched for feedback. Extensive use was 
made of buttons, scrolling fields, printing fields, varied 
text size/fonts/style, clip art, scanned graphics, screen 
captures and draw tools. Of the slacks that used 
graphics, the most frequent number was one though the 
one may have been repealed on several cards. Some 
stacks used as many as 38 different graphics. Many 
stacks had no way to orient the user to the sequence of 
the present card, but the most often used technique was 
a card label. Most slacks appropriately arranged the text 
and card display. 
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Table L Analysis of HypoCard Stacks 

1. Number of HyperCard Stacks: Phase I = 30. Phase II = 70, Total =100 

2. Curriculum areas: 

Saatct Math Language Social Studies Computer Other 

17 20 22 3 34 4 

3. Average number of different navigation button messages in stacks: Phase I = 5; Phase II = 4 

4. Other features used in stacks 
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Some used straight text that gave a crowded and messy 
appearance. Some typed text in all capitals. 

The novice authors, by and large, created programs ihal 
were amazingly good for the amount of training given. 
The stacks from Phase II were even better than those 
from Phase I. This may have resulted from trainers who 
now hr.d more expertise with HyperCard and could 
demonstrate and follow up with more examples and 
techniques. 

During about the same time period in 1989-90, two 
consecutive university courses focusing on HyperCard 
were offered. A core group of teachers took both 
additional courses. In the second course the experienced 
authors were joined by a group with no previous 
training on HyperCard. The focus was instructional 
design and screen design, scripting, development of 
visual, audiovisual and video disc stacks. During the 
second course, there was a great deal of collaboration 
and critiquing of stacks among all students. The 
resulting 46 stacks fr^'** the two courses were also 
analyzed There seemc ^ ' be little difference among the 
two groups in the sec;. * ;oursc in terms of quality of 
Slacks and qrplicationf cl HyperCard techniques. 

The stacks by experienced authors were typically more 
complex than those by novice authors. Experienced 
authors' stacks showed greater application of 
instructional design in the way they were designed and 
executed. More sophistication was evident in screen 
design and lay out Creativity seemed to be a larger 
factor in their stacks. But with additional instruction, 
experience and particularly time, the novice authors 
could reasonably expect to produce programs, on a par 
with those of the experienced authors. As with most 
things, experience shows. 

Evaluation of HyperCard Stacks 

Developing CAI programs is just an interesting, though 
time consuming hobby unless those programs are 
useful and the development process adds to the author's 
growth as a teacher. To assess the usefulness of the new 
stacks, the CES teachers reviewed and rated them. Their 
purpose was to select stacks that could be reHned and 
included in an Instructional Technology Guidebook for 
disuibution to al! high schools. 
Of 146 stacks, 95 were selected for inclusion in the 
Guidebook. Most needed some degree of revision and 
some needed extensive revision. 

The stacks were edited and annotated by a group of 15 
teachers who were identified as excellent HyperCard 
users. Most of them had participated not only in the 
initial training but also in the two courses. They used a 
checklist to examine and revise each program to meet 
the criteria. 



Figure 1. HyperCard Chcc'^list 

STACK 

□ has been compacted 

□ includes an ovcrview/inutxluction 
Q includes an objective 

□ includes directions as needed 

□ has a menu allowing nonlinear access 

Q clear focus and logical progression of program 

□ performance measure of leaniing 

□ content, spelling, grammar correct 

CARDS 

Q includes a title card 

Q each card has navigation options 

Q adequate white ^ce 

□ one idea per card 

Q appropriate graphics are used 
Q text is in Ttelds 

Q text size, style, and font is appropriate 

FIELDS 

Q appropriate font options 

Q text locked when not to be changed 

Q wide margins for text 

□ scrolling Held rather than card full of text 
Q one idea per Held 

BUTTONS 

Q use same locations for same buttons 

Q uniform styles 

Q work as intended 

Q purpose clearly indicated 

Q no exuaneous buttons 

BACKGROUNDS 

Q appropriately used 

The most frequent revisions and additions were ways to 
check responses, more graphics and examples, breaking 
up dense text loads, using more interesting instructional 
su-ategies, tighter focus on the program objective, 
increased interactivity, printing out test results and a 
comment card. 

Interviews with the teachers revealed some authoring 
tips they learned through the experience. If a script 
doesn't work at one level, it may at another level. Use a 
conservative approach and change only one variable at a 
time. It may affect something somewhere else in the 
script. Compact the disk and si c information 
frequently. Have a back up disk in case of disaster. A 
faster machine makes development easier, but keep in 
mind the end user machine. Think about whether to use 
the Macintosh as a PC or to run the stack through the 
network server. There is a time variable. The PC gives 
consistent rate but the server may slow down when 
dealing with several requests. For some stacks, this may 
make a difference. 

The work of many teachers has resulted in the 
Instructional Technology Guidebook. It provides a 
wealth of material for use by teachers and student.^. 
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Included are diskette copies of the refined HyperCard 
stacks as well as examples of other software tools that 
were part of the New Kid Graduates training. The 
software programs are presented as part of a larger 
instructional design. Each is described by topic; 
expectation; content areasAhinking skills; intended use 
by user and by environment; enabling activities for 
teacher and student; hands on activities; follow 
up/feedback activity; evaluation of student learning; 
application type. 

Impact of Authoring 

To assess the impact of the a-^thoring activity on stack 
developers* a questionnaire was given to the IS teachers 
described above. While probably not representative of 
the total group of authors, their reactions provide some 
insight into the value of their involvement 

They indicated that during the New Kid training they 
had not learned much new information about computers 
but did leam a lot about HyperCard and computer 
authoring. They learned about instructional design and 
screen design as well as suitable instructional 
approaches for *jse in computer programs. All indicated 
that the process of working with HyperCard has added to 
their professional growth and most felt it has also added 
to expertise wit' their curriculum. These are teachers 
who have mastered a new area that they will continue to 
use for the benefit of students and other teachers. 

Conclusions 

Teachers can become HyperCard authors and still have 
time to be teachers. They can develop stacks to teach 
their curriculum As a groups they can devi;^ stacks that 
will cover many important areas that are taught year 
after year.^ Expertise with computu based curriculum 
development tools empowers teachers io use technology 
more appropriately and successfully. 

Specific conclusions are as follows. 

1. Given a minimal amount of training (12 
hours) and development time (40 hours)» 
secondary teachers who are novice HyperCard 
users can develop simple CAI programs 
suitable for use in the content areas they teach 
or for training other teachers on use of the 
technology. 

2 . Given more extensive instruction and/or follow 
up support* teachers can develop fairly 
sophisticated stacks using visual* audio-visual 
and interactive video fcKxnats. 

3. The differences between novice developer 
stacks and experienced developer stacks lie 
primarily in the use of instructional design, 
screen design and layout and in creativity. 
These are largely factors of training, experience 



and time. The creativity is shown in novel 
ways to present ideas visually. The impact of 
print versus graphic visual presentations is 
vividly demonstrated in a stack. 

4. Teacher develq)ed stacks are useful, approprhtt 
and suitable for use by teachers and students. 
Those programs can be used by other teachers 
for instructional purposes and as models for 
additional HyperCard stacks. 

5. Teacher authors indicate that they experience 
professional growth as they combine a CAI 
authoring tool, instructional design and their 
curriculum area. 

6. Teachers can share this tool with their students 
who use it to develop their own programs. 
When students present learnings in an 
instructional format, they learn them better. 
Student auth(Hing of HyperCard stacks can 
offtT divergent and creative paths to improving 
writing skills, deve]q)ing thinking processes 
and self expression in different modalities. 
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Doris Smith Shaw, US Army Construction Engineering Research Laboratory 



Abstract 

Research at United States Amy Construction 
Engineering Research Laboratory (USA-CERL) has 
contributed some new information to the issue of locus- 
of-control in Computer-based Training (CBT). 
Architectural and engineering designers indicated that 
they favored options which pcvmiMcd them to explore 
the training materials without computer intervention. 
The users also indicated that they preferred on-the-job 
support as opposed to a purely training sessioa Hyper. 
Text-type environments are often suggested to provide 
resource support mat may be controlled by the user. 
User control offers flexibility and adaptability to those 
with a wide range of learning styles, objectives, and 
entry knowledge, as was found in our test sample. 
However, control of the learning environment by the 
student can be confusing and frustrating, especially for 
new trainees. Structuring the knowledge with awareness 
of user modes and tailoring the interface to correspond 
to the structure is an approach to the problem. Our 
prototype system employs this meihodoiogy. 

Background 

Research on computer-based instruction for computer- 
aided design (CAD) software began at USA-CERL in 
1985. Corps of Engineers architects, engineers, and 
drafting personnel tested interactive embedded 
instruction for the AutoCAD software package. Test 
subjects completed the computer-based course of study 
and were tested for knowledge of the CAD system. Our 
findings were reported to ADCIS (Shaw, 1988). We 
then conducted follow-up interviews to investigate some 
of our hypotheses concerning the research results (Shaw 
andColish, 1989). 

The interviews vith test subjects brought up new issues 
that should be considered by those who train adult 
professionals. Factors of individual variation in 
objectives, learning style, and resource needs suggested 
additional instructional system requirements. 
Questionnaires and interviews also disclosed that some 
features of our earlier computer-based training (CBT) 
were inappropriate for the intended audience. Some of 
the problems leading to new system design stemmed 
from traditional CBT approaches and some from the 
needs of the target audience. 

Problems 

Traditional CBT. The tutorial displayed reminders 
to practice the exercises and gave feedback if expected 
errors were made. These features were either disliked or 
were considered unnecessary by many of the users. 
Eighty-six percent agreed with the questionnaire item "I 



usually knew when I did things wrong in the lessons.** 
Twenty-eight percent indicated that the reminders 
annoyed them. They remarked that they would practice 
if they liked and that they usually could see if they were 
correct or rK>t as they woiked with a drawing. 

Users did indicate that most of them preferred learning 
about CAD from interacting with a computer rather 
than studying it from a book. However there was 
agreement that they knew what they needed to learn and 
appreciated help in reaching answers to specific 
questions. The index and the freedom to explore the 
command menus were very popular features. 

In general, when the user controlled the learning it was 
the most effective. These learners preferred not to be 
subjected to many of the most common CBT design 
features. The reasons for this are rooted in the 
differences between adult professionals and the school 
children who have been the test subjects for most 
traditional CLT. 

Audience Needs, We discovered that the best CAD 
users were required to use CAD intensively in their 
work and were "discovery" oriented in their learning 
style. These users tended to have specific objectives in 
mind and tested their own ideas as they worked. They 
often skipped around in the lessons by me'^ns of the 
index and tended to learn the parts of the system that 
were most closely related to their own work. Tnose 
concerned with file transfer noticed "clean" databases and 
the size of the file, while those concerned with accuracy 
pi'^ked up rne information about numeric input. Users 
might not notice features which they did not meet in 
ihe.r work. 

In many of the interview situations, skills and 
tec^niques were observed that were not part of the 
lesson objectives. Users would demonstrate iterative 
features and programming routines that had not been 
specifically taught. Setting lesson objectives for these 
sophisticated students was a questionable process. 
Outcomes, particularly student-specific ones (Eisner, 
1985), should be a main concern in training for adult 
professionals. 

It was common to hear that learners had reached a point 
in the lessons at which they felt prepared to apply their 
learning to a ^\ ork project and had left the training for a 
period of time. Many times they returned to the lessons 
after completing a production design. They would 
explain that they understood more after working with a 
"real" problem. They would then seek out areas of study 
in which they realized they were weak. Apparently their 
metacognitive capacities developed from on-the-job 
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experience and could be trusted to guide their learning. 
As has been observed in other current research 
(Steinberg, 1989)» these users developed learning 
strategies as they be^:ame more aware of CAD concepts. 

Another problem to be considered is that different types 
of representations are more instructive for people of 
different cognitive learning styles. This was particularly 
observed in this audience of architects, engineers, and 
drafting personnel. While some learn through 
abstractions (words), others may profit more thr^ Jgh 
imagery (pictures), activity (practice), oc concretizing 
(examples) (Maytr, 1983). At times it may require more 
than one type of representation to illuminate a concept 
and that may depend upon the concept itself. The test 
(of concept specifications)... should focus on the 
, eciflc functions of a particular concept (Engelmann, 
1969). 

Some of the test subjects expressed the need for having 
the lessons present while they worked on a design." 
Ideas flow better with facts at our fingertips-fewer 
intemiptions for lodc-ups, and fewer detours down blind 
alleys" (Penzais, 1989). Others indicated that the 
lessons took too much time from their jobs and that it 
meant more when they could apply the information as 
they worked. They preferred to seek specific topics to 
study as the need arose, rather than set aside a penod of 
time for training. This finding prompted us to consider 
Compuler-Based Training (CBT) with a higher degree of 
user control than we had previously used. The Hyper. 
Text approach was an obvious possibility. The problem 
was to design a concurrent instructional system which 
would provide for user differences in knowledge and 
objectives. 

System Requirements 

Some of the requirements of the new system came 
about as a response to the tests and interviews. 
Eexibility in mode of use was specifically aimed at the 
adult professional audience. Other requirements grew 
from our increased study of information processing. 
User control of the environment presents some new 
problems in our high-tecii *vorld. It is relatively simple 
to arrange a snudl body of information and provide 
computer access to it, but it becomes a very difficult 
task to do the same thing for a large, complex, and 
possibly interrelated body of infonnation. We are told 
that soon all documents that are valuable enough to 
keep will be available electronically, which is an 
overwhelming idea. Levels of abstraction and methods 
of classification become extremely important issues. 

The first requirement was that the training should run 
concunently with a drawing and remain m view while 
the drawing is active. On most large workstation 
screens, it might occupy a part of the same screen or a 
second monitor. Even on a PC workstation, the user 
could swap back and forth between the training and the 
design file with a single keypress. Design files 
especially prepared for each lesson would be developed. 



Structuring Knowledge for HypeiText 

They would contain drawing eleir. that directly 
correspond to the practice exercises. Most of the 
exercises could be practiced in any design file. The 
system might then be used with a lesson drawing, a 
blank drawing file or a "working drawing" in progress. 

Different modes of use that should be accommodated 
might be: 

1 . Overview 

2. Sequential 

3. Discovery/browsing 

4. Look-up 

"Overview" mode is a common user mode. A manager 
might wish to scan the content to get the basic concept 
infonnation, but not examine procedures or examples. 
Learners often wish to preview or review infonnation. 
Some students desire a sequential pattern when the 
subject area i. new to them. As they become more 
familiar with the topic, they often develop their own 
objectives and wish to explore specific lines of thought 
in a "discovery" mode. "Sequential" learners would 
probably use the prepared examples and practice ex- 
cises. Those who are more discovery oriented might 
a CAD drawing that they are working on, or might 
d^gn exercises for themselves. In "look-up" mode, the 
user would be in pursuit of specific procedures or details 
while working in a draw'ig file. 

Several different types of relationships between 
information must be established: 

1. An implied linear tq}ic sequence 

2. Capability of cross referencing in a Hyper. 
Text-like mode 

3. Several different possible organizational 
patterns for accessing the information, among 
them priority of usage, category of operation, 
continuum of procedure, and alphabetical 

4. Different types of information (pictorial, 
example, practice, explanations) related to the 
topics when 2^)propriate 

5. Direct links to related topics 

6. Provision for the system to link to the 
growing body of documentation and 
applications needed in the future 

The linear sequence makes starting points and next steps 
obvious to learners. It solves the problem often found 
in documentation when there are several manuals 
included or where the information is alphabetically 
ananged. User observations and assumed entry 
knowledge determine the most appropriate sequence. For 
example, in our introductory MicroStation instruction 
DELETE ELEMENT is placed before PLACE 
ELEMENT because users were frustrated when they 
drew unsatisfactory elements and did not know how to 
erase them. Also, PLACE FENCE should be taught 
before CREATE CELL because a fence must be placed 
around the cell elements before the complex element 
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H MORE INFORMATION 
)G EXAMPLE 
Hi PICTURE 
ic PRACTICE 
^ PROCEDURE 
II OTHER 

fnnding Monntti< 



TWO WAY CROSS REFERENCES 

WITH BACK ARROWS 




Figure h Knowledge Stmciure Diagram. All information screens provide a return to the MAIN INDEX. Those 
arrows have been omitted for the sake of clarity. Cross references always offer a return to the point of reference. 



may be created. However, all users will not use the 
sequential oider if they are permitted to do otherwise and 
cross references should be provided to help them fill in 
information gaps if that is needed. This is one of the 
greatest powers that computer-based systems have over 
books. In a book, there is an implied sequence, but 



usually no provision for filling in skipped information 
other than using the index. Hyper-text approaches make 
such references easy. 

Images of buildings might be presented geographically, 
historically, by architect, by view, or by function. Each 
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Structuring Knowledge for HyperText 



method of arrangement offers new information and suits 
a different purpose (Wurman» 1989). Topics may be 
related* Ixit not dqpoidem upon each other. T!i^ 
be provided with direct liniks* Alphabetical command 
refmncing, available in many on-line he^ programs* is 
effidem when the command name is known. Otherwise* 
it may be likened lo searching for an unknown name in 
the phone book when only the address is known. 

The user interface for the system also has several 
requirements: 

1. Be intuitive and require no additional 
instruction 

2. Present complete information in each view 
without scndling 

3. Have some constraint upm the size of 
infocmadon chunks that Ae user must digest at 
onetime 

4. Retomio the previous location at all times 

5. Related knowledge clusters be associated with 
no more than four keypresses if possflde 

6. deariy libeled access to related information 
and an obvious return provided to the original 
reference point after GI08S refisfcooog 

7. Diffioemns^ge modes be obvioos to Ae user 

8. A system map wUch pennits users to chart 
their progress and acc^ other parts directly 

9. Provide nserivogiamnuble reference links 

Scndling screens, even when they scroQ both down and 
up, 9tt distracting and hard to assimilate. If additional 
informatioa is needed, a keypress shook! show a 
diffeitm page «d its relationship 10 fl>e first dioold be 
labeled. Return to Oe previous screen sfaookl be an 
option fipom all screens. From cross references, a simple 
back-arrow must be provided, since cross referencing is 
ofien not meant to shifk atlemioo finom the user's main 
objective. The scrolling screen constraint serves to 1^ 
a focus in conc^ presentation^ however, the size ot 
larger knoiriedge blocks must also be controlled and Ae 
relatioodi^ between them clearly defined. Mediods 
sudi IS annotated tables of contents and monitoring user 
activities should aim at quick access to the desired 
information. Pricmtizing among the available options 
may be possible through intelligent programming 
flp prffli ch cg. 

It is important that users be able to track their 
performance through the computer information, if a 
map is availaUe diat shows the current tocation in the 
knowledge structure and what selections have been 
viewed* users can orient themselves. If a uso* decides to 
change modes, for example from sequential to 
discovery, it shotdd be easy to make the change. 

Mapping Information Content 
In order to visualize tiie structure of the system* a 
knowledge cluster diagram is used. For each content 
node there are entry and exit arrows to otiier* more 
general* more specific* or parallel nodes. These are 
clustered acccvding to category of operation. Cross 



references are provided to procedures or terms which 
may be needed to use or understand a given explanation 
if a student has followed other than tiie implied 
sequence. These are indicated by dotted lines. Different 
types of information are represented by icons 
overflying the topic to which they src iclaied. Arrows 
will sot uc drawn from these icons in the interest of 
simplicity. 

Rgure 1. is a graphic representation of the links 
between information clusters. It shows the structure of a 
small portion of the knowledge c^ered in a prototype 
concurrent embedded support system for tiie Intergnph 
MicroStation CAD system. This segment covers 
electronic / hardcopy media transfer, specifically trace 
digitizing and plotting. The selection is relatively 
simple* (rffering five levels of generalization at tiie 
most It is a small partof the entire system. 




Figure 2. Overview 
User Paths 

Users of seveial types can be expected to (^Kxate such a 
system. Differences in objectives as well as learning 
styles can be accommodated. The practice suggestions 
may be carried out in a special design file prepared for, 
tiiat purpose or in a production drawing in pn^gress. The 
large chart shows the total structure. The small charts 
tiuit follow show the paths selected by typical users. 
Refer to the large chart to read the text The user in 
Kgure 2. has elected to review just tiie main topics in 
the stiructure. It is probable tiiat ibt infcmnation needed 
is ratiier superficial. Perhaps this is a manager or 
someone trying to obtain an overview. In tiiis case* 
tiiere may be no design file for practice* since tiie 
practice exercises are not viewed. This is a qukk way to 
become familiar wi\h the general coverage of tiie 
subject The way to use tiiis mode is. to select tiie 
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NEXT TOPIC button. LAST TOPIC would show the 
same thins order. See Figure 7. for the 




Figore 3. Sequential Path 



The user in Kgure 3. has progressed sequentially 
throQ^ all of die infcnmation screens. The user is 
possaUy a structured kanier who doesn*l want to miss 
anydiing. Probably this user would be working from 
the spedal design file and would likely try the piactice 
exercises. NEXT CARD LAST CARD would be 
employed to work through the instruction in diis mode. 
SeeHgure?. 

The user in Kgure 4. has used the indices to reach 
selected topics and pursue some of them in detail Cross 
references have been used and some practice exercises 
have been viewed. The user may be using either the 
^)ecial design fik or a production drawing. This type (rf 
usage indicates a motivated user, {Mobably discovery 
oriented. Most professional designers and draftsmen 
exhibit this typt of path. At the beginning th^ may be 
structured learners, but they quickly become more 
discovery oriented. 

The user in Figure 5. has Doved to the annotated table 
oi contents to k)cate the particular information neeced 
and has gone directly to die stream mode panuneters. 
This Qrpe of path would be typical of the experienced 
user who needs ^)ecific suppcHt for his production. This 
user is 

Many on-line help systems provide a browse mode 
which is helpful for the discovery learner. Such 
behavior on the part of the user assumes some famil- 
iarity with the subject The major differences in our 
system are the overview and sequential paths that are 
clearly indicated. Look-up mode is also available in 



many systems for the user who knows what he wants to 
find. Htowever, a great many systems do not allow users 
to view the he^ while carrying out die instruction their 
own projects. Many times a user must copy the 
information from a detailed he^ screen in order to keq) 
it in mind. 




Figure 4* Discovery Padi 




Figure 5* Look-up Mode— the one who would make 
die most use of alphabedcal organization. 
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The Interface 

The interface must be constructed with the knowledge 
structure in mind Finding the oxrect content and type 
of information must be intuitive and efficient We used 
a TileCard" met2q)hor. Cards are arranged in boxes 
which represent menus. You may click the mouse or 
digitizer on the tab to read the card. See Rgure 6. 




•KUN toot 



Figure File Box 

A learner may move throu^ all of the cards by 
selecting the NEXT CARD button or backwards 
sequentially by selecting LAST CARD. If only the 
general topics are desired^ NEXT TOPIC tx LAST 
TOPIC will present that type of overview. Cross 
referencing is also possible. Clicking the mouse or 

digitizer on the symbol, jumps directly to the 
infcvmation about DIGITIZER SETUP. The return 
anow provides a return to another cross-reference page. 
Seel^gureT. 

Buttons on the bottom right-hand edge of the card relate 
to the same topic Index buttons are at the top left edge 
and sequential buttons are at the lower left-hand comer. 
NEXT TOPIC and LAST TOHC cards appear on the 
TOPIC CARD for each section. "Design the set of 
alternative methods for a task so that the rule for 
selecting each alternative is clear to the user and easy to 
iVldy** (Card, Moian, and Newell, 1983). 
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Figure 7, Topic with cross references 



TOPIC CARDS present genml explanations. MORE 
INFORMATION cards usually go into more detail 
about the concept PROCEDURE and PRACTICE 
cards are used when needed Practice cards suggest 
exercises in the jHepared drawing file. Pictures and 
examples as shown in Figures 8 and 9 are essential for 
some learners and are gencraDy included Other types of 
cards may be employed for special purposes, such as 
NOIES cards to accompany the table erf contents. 
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Figure 8. Concq>t Illustration 



The card size Umits the amount of information in each 
chunk* If more space is required the nature of the 
relationship nuist be defined 

There are no requirements on type or number of 
information cards for a topic, however the space for 
buttons tends to limit the treatment to those with 
highest primty. If more buttons seem to be necessary, 
a sub-topic should be considered. In this way, 
relationships are clarified and boundaries are set Also, it 
is always possible to add new information when the 
stnicture is expanded. 



System Consistency 

Experience over the last five years i»oves that frequent 
changes can be expected in both hardware platforms and 
software systems. 

Users may use many computers, based upon 
availability, application, and specific work assignment 
Portability between environments is a requirement for 
text files, drawing files, and databases. Training and 
support systems should aiso be portable and as 
consistent as possible. 
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Figure 9. Example 

Users should not encounter frustration when they 
change machines. While certain applications packages 
and hardware may offer additional powers, these should 
be viewed as enhancements, not learning problems. The 
concurrent support system designed at USA-CERL is 
able to accommodate this need. With this capability, 
instruction for one computer system could be referenced 
from a different computer. At times, this might be the 
most efficient use of resources. 



Conclusion 

User controlled, HypcrText-typc support systems offer 
the flexibility and adaptability that adult professionals 
require. Carefully designing the knowledge structure and 



the computer interface with user modes in mmd is one 
way to provide freedom, as well as a control mechanism 
to the user. Consistency between different platforms of 
the same software system should be a general objective 
in CBT. 
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PROTOTYPING INTERACTIVE MULTIMEDIA EXHIBITS FOR 
MUSEUMS: A CASE STUDY 



Brockenbrough S. Allen, San Diego State University: 
and Nanette T. Sterman, University of California San Diego 



Abstract 

This case study describes the application of interactive 
multimedia technology and rapid prototyping 
meii.odology to the development of exhibits for a new 
aquar^am-museum under construction at Scripps 
Institution of Oceanography. Enveloped by graduate 
students in educational technology at San Diego State 
University, the prototype exhibits reflect diverse 
paradigms for informal learning and content ranging 
from shark physiology to ocean climate patterns. The 
prototypes will be used to demonsu^tc the potential of 
interactive multimedia exhibits and to solicit funds for 
development of full-scale versions. Rapid prototyping 
enables exhibit curators and designers to represent 
proposed designs in forms that are tangible enough to 
permit evaluation by sponsors and administrators as 
well as by representative visitors. 

Introduction 

What is a museum exhibit? A plaque with text and 
graphics? Silently staring animal specimens? Artifacts 
encased in glass? For centuries, museums have 
preserved and displayed real objects. Now computers, 
video, and interactive multimedia have expanded the 
options for visitors to explore and interpret museum 
collections. New media-based paradigms such as 
hypermedia and hypertext offer conceptual pathways that 
can make museum experiences more meaningful. 

There are two reasons that museums have been slow to 
adopt interactive multimedia. Few museum specialists 
are aware of the rich and dynamic communication 
potential of interactive multimedia technologies or how 
they can be implied to make museum information more 
accessible and useful to the public. Most museum 
specialists do not have enough experience with these 
new media to visualize complex computer-based designs 
from paper-based specifications. 

The second reason that interactive media are not yet 
prevalent in museums has to do with money. Securing 
funding can be particularly difficult with interactive 
multimedia exhibits-unless the potential sponsor can 
see an actual example of the proposed project. 
Therefore, one of the first challenges facing would-be 
developers of multimedia exhibitry is to represent 
proposed designs in forms that arc tangible enough to 
permit evaluation by sponsors and administrators as 
well as by museum visitors. 

This case study examines how HyperCards-based 
interactive video systems can be used to create 



prototypes ^propriate for presentation to a variety of 
audiences. It also describes how graduate students in 
educational technology can contribute to the 
development of interactive multimedia exhibits and how 
r^id prototyping can serve the needs of clients as well 
as student designers. 

Origins of the Project 

In the Spring of 1989, the authors entered into a series 
of discussions with staff from Scripps Institution of 
Oceanography Aquarium-Museum at the University of 
California San Diego. Those discussions lead to the 
creation of four prototype interactive videodisc exhibits 
for the museum portion of Scripps' new aquarium- 
r:*jseum facility. The prototypes were researched, 
written, and produced by graduate students from the 
Department of Educational Technology at San Diego 
Slate University in collaboration with content 
specialists firorn Scripps. For descriptioiu*^ of the SDSU 
educational technology curriculum on interactive 
multimedia* see Allen and Erickson (1986) and Allen, 
Dodge, and Saba (1990). 

Scripps Institution of Oceanography (SIO) Aquarium- 
Museum in La Jolla, California, hosts nearly 4(X),(XX) 
visitors each year. Presently, the Aquarium-Museum is 
in the planning and fund raising stages for an aquarium- 
museum facility to replace their aging, 40-year old 
aquarium. The new 32,000 square-foot facility will 
enhance the aquarium's ability to meet its major 
mission: interpreting research done at Scripps 
Institution of Oceanography for the public. 

Visitor research conducted in 1989, showed that the 
Aquarium-Museum serves three audiences: 

• Organized groups, mostly classes of school 
children, who come to the Aquarium-Museum 
for a one-hour guided tour, conducted by a 
volunteer interpreter. The interpreters tend to 
be retired individuals who are trained in the 
basics of marine science by the Aquarium- 
Museum education staff. 

* Local families who visit the Aquarium- 
Museum on the weekends as an entertaining 
and somewhat educational experience for adults 
and children to share. Overwhelmingly, these 
small groups come to see the variety of fishes. 
Out-of-town visitors who have heard of SIO 
from friends or family. This group also comes 
primarily for entertainment, but like the family 
groups, many express considerable interests in 
aquariums, in fishes, and in ocean-related 
hobbies. 
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The Scripps staff was interested in developing 
interactive exhibit prototypes for two reasons. First, 
they wanted to explore new ways to help the public to 
interpret oceanographic research. Second, they sought 
inexpensive prototypes that could be shown to potential 
sponsors and ager)cies who might fund the new facility, 
due to open in 1992. Since this was the Aquarium- 
Museum's first experience with technology-based 
exhibits, the staff was open to development of several 
different prototypes-each exemplifying a different mode 
of '..leraction or a diffe' nt approach to visitor 
exploration. For the graduate students in ''Educational 
Technology 653: Interactive Video Instruction," the four 
prototypes represented an opportunity for "hands-on" 
development of interactive multimedia programs 
intended for informal learning environments. 

Project Goals 

A major goal of the Scripps prototypes was to 
demonstrate how interactive multimedia exhibits could 
be used to help visitors acquire information ^ their own 
pace and integrate it with other museum experiences. 
Several factors were considered during early planning 
stages; among them were Scripps' concerns regarding: 
specific subject matter related to the Aquarium- 
Museum's mission; 

trafHc panems and visitor flow through the 
museum; 

• visitor interactions limited to two or three 
minutes at each interactive exhibit 

Audience analysis is critical to the success of any 
museum exhibit The design of interactive multimedia 
exhibits for a museum environment must take into 
:ount the typical Scripps visitor who is likely to be: 

a first-time user; 

relative! y uninformed; 

of various ages, nationalities, and educational 
levels; 

reading at a level about 7th grade equivalent; 
anxious about an unfiuniliar medium; 

• spending less thaii a minute at each exhibit 

A second goal of the project was to show how 
interactive multimedia might be used to increase exhibit 
"holding power^ which in the case of traditional exhibit 
technology is generally low. Clowes and Wolf (1980) 
reported that visitors at one museum spent only 17-23 
seconds looking at dioramas. According to Van Rennes 
(1981), "in most museums, visitors spend only an 
average of 45 seconds at any one exhibit" Observations 
of visitor viewing patterns at Scripps indicate similar 
patterns: viewing times for current exhibits cluster at 
seven seconds and seventeen seconds. 

Interactive multimedia exhibits rely on diverse 
paradigms for structuring information, rangmg from 
architectural/spatial (Sircus, 1989) to taxonomic 
(Diamond, Bond, & Hirumi, 1989). Interactive 
multimedia posed a challenge for the museum's 



professionals who were unaccustomed to thinking in 
terms of diverse and simultaneous structures. On the 
other hand, it also challenged the students whose 
graduate training had emphasized mastery-oriented 
instructional design. Informal learning is intended to 
entertain, to inspire, to captivate and to raise ciinosity 
without focusing on measurement of explicit learning 
outcomes. 

Museum exhibits present unique requirements for the 
^plication of interactive multimedia technology. For 
example, one of the most difficult challenges is the 
^propriate usage of audio. Transient sound effects and 
short audio segments are acceptable, but long audio 
passages often interfere with the experiences of museum 
visitors who are not observing the videodisc exhibits. 
Several exhibits with significant amounts of audio can 
result in a tangled audio mess. To accommodate this 
constraint, three of the prototypes are primarily text- 
based. (The fourth is intended f(»: use in a classroom 
where noise limitations are not a concern.) 

To capture visitors' interests and curiosity without 
relying on audio, exhibit texts are written in an 
interpretive style. Interpretive writing strives to link 
new concepts and information to the visitor's prior 
experience and knowledge base. It uses simple 
descriptive language to suggest visual images. It also 
uses questions as a tool to stimulate visitor 
participation in the exhibit 

Rapid Prototyping 

The graduate students used rapid prototyping 
methodology to develop working versions of the 
exhibits. Interactive multimedia programs integrate 
diverse communications modalities (ranging from 
speech uid music, to photos and motion pictures, to 
text and diagrams) in complex and dynamic 
relationships. Hence, environments for developing and 
exercising interactive multimedia prototypes must be 
abie to demonstrate how these modalities will be 
coordinated. Furthermore, the prototypes must 
communicate to decision-makers and representatives of 
the visiting public who may have a limited ability to 
visualize designs represented through means such as 
narrative descriptions, flowcharts, and storyboards. 
Rapid prototyping — a term borrowed from software 
engineering-refers to ". . . system development 
methodology based on building and using a model of a 
system for designing, implementing, testing, and 
installing the system" (Lantz, cited in Tripp r-Tid 
Bichelmeyer, 1990, p. 35). Rapid prototyping requires a 
computer-based development environment that permits a 
modular approach to development and allows software 
to be edited and reshaped quickly. On the other hand, the 
development environment must be disciplined enough 
to permit logical definitions of the prototype and 
physical representations tnat are stable enough so that 
they can be exercised or tested. 
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Finally, there is the design process itself. Since exhibits 
are rarely expected to promote specific learning 
outcomes, it is more useful to think of them as part of 
informal learning environments rather than as devices 
for instruction or training. Systematic approaches to 
design in which exacting definitions of learning 
outcomes precede prototype construction may be less 
effective than more open and intuitive approaches in 
which exhibit designers refine their proposals for 
learning outcomes following direct experimentation. 

As a paradigm for exhibit design by graduate students in 
educational technology, the potential advantages of rapid 
prototyping with HyperCard^ are four-fold: (1) it 
permits student designers to explore alternative designs 
with relatively modest investment in programming; (2) 
it allows them to communicate the **look and feel** of a 
proposed product to reviewers and decision makers; (3) 
it encourages early formative evaluation with 
representative users; (4) it serves as guide to completion 
of a final version which may be developed and delivered 
through systems other than HyperCard™ and by 
specialists other than designers. A special advantage of 
HyperCard''^ for rapid prototyping is that screen 
displays and user controls can be quickly and easily 
modified by designers who have little programming 
experience. The ease of modifymg HyperCard™ 
prototypes makes detailed paper-and-pencil designs less 
important, perhaps changing the need for rigid 
approaches to design documentation. 

Project Development 

Several months before the course began, the authc^ laid 
groundwork for the project by conducting an informal 
analysis of Aquarium-Museum exhibit needs and by 
developing tentative formats and topics for the exhibits. 
Twenty-five graduate students, two professors of 
educational technology, the aquarium-museum director 
and the curator of education were the primary 
participants in this project. Others included subject- 
matter experts from Scripps, San Diego State 
University, and private research companies. The 
students were split into four working teams. Within 
these teams, individuals were designated as specialists: 
instructional designer, video producer, graphic artist, 
etc. Two students were designated as class managers, 
one specializing in the technology and the other as a 
subject-matter/museum specialist and liaison with 
Scripps. 

The development budget was restricted to 52,400 of 
which $1,000 was allocated to mastering two copies of 
a 54,000-frame videodisc. Each te::m was allocated $50 
for out-of-pocket expenses. OlJier resources were loaned 
or donated. 

Individual tcam^ researched the content of the exhibits; 
created the program specifications; and amassed 
resources and materials including photographic images 
from Scripps archives, video clips from a local 
television station, and original video shot for the 



project. Though planning and design documents 
specified complete exhibits, the prototypes were partial 
implementations: some on-screen buttons were 
"dummies," representing options for later development 

Description of Prototypes 

The four prototypes are quite different from each other. 
They deal with different subject matter, uses different 
navigational systems, and employ different types of user 
interactions. Brief descriptions follow. 

Exhibit 1: The Fish Encyclopedia uses an 
existing videodisc and is an example of a low-cost 
approach to exhibit development. Graduate student Tom 
Dwyer developed PioneerScripter, a HypciCard™-based 
authoring system for generating Level II videodisc 
control programs and downloading them into a Pioneer 
LD-V6000 player. PioneerScripter eliminates the need 
to program the LD-V6000 with a cumbersome key pad. 
With PioneerScripter, keying in a program can take less 
than an hour and downloading is nearly instantaneous. 
Control sequences can be saved on diskette for future 
use. 

Twelve examples of underwater animal life were selected 
from the fcny-five species featured on a cotTtmcrcia! 
videodisc entitled "Fish and Invertebrates** (Pic^.eer 
Electronic Corporation, 1988). These examples were 
grouped in the following sections: "Fish Reproduction,** 
"Fish Feeding,*' and "Fascinating Fish.** 

The Fish Encyclopedia was designed for viewing on a 
large screen built into a wall of the Museum- Aquarium. 
The control panel is to consist of five durable push 
buttons. Instructions on the screen gui le the visitor 
tljrough the steps. The exhibit can be used in three 
ways: 

The visitor can view the exhibit passively by 
watching a continuous introduction loop. 
The visitor can select a button which will take 
him/her to a set of four still photographs of 
fish. 

The visitor can schxt a third button to learn 
about any of the foui %h. 

Exhibit 2: Exploring an Oceanographic 
Research Vessel takes the viewer on video tours of 
research ships in the Scripps fleet (See Figures 1-7). 
This exhibit is intended to give visitor an impression of 
"life at sea*' and of research aboard ship. Viewers select 
locations on a schematic of the ship: the engine room, 
living quarters, bridge, main lab, and after deck. In each 
location viewers observe the scientists and the crew 
members at work. 

Scripps operates three basic types of research vessels. 
The vessel chosen for emphasis in the prototype was 
the R/V New Horizon, a typical surface ship. The 
prototype also includes brief introductions to RV FLIP 
(Eloating Instrument £iaiform) and the RV ORB 
(Qccanogiaphic Ecsearch Buoy). 
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Most of the footage for the prototype was shot 
documentary style. Live-action sequences and 
photographs were acquired from existing film 
documentaries. Footage shot aboard the New Horizon 
by team members was also incorporated into the 
program. People seen in the New Horizon video 
segments are actual ship personnel and team members. 

Exhibit 3: El NiRo explores an important and 
dramatic climatic phenomenon in which the flow of 
important ocean currents is reversed, resulting in odd 
and sometimes destructive weather patterns. Viewers 
follow a story line to learn about El Nifto: its 
characteristics and range, iiow it effects ocean lire;, and 
how it appears in satellite photos. 

Audio and text displays offer a general overview of the 
£) Niflo phenomenon. Visitors can use this information 
to predict the effects of El Niflo on tuna populations and 
then manipulate ocean water temperature to test their 
predictions. Live video images (obtained from KFMB, a 
local commercial television station) show windstorms 
and coastal flooding caused by the 1983 El Niflo. 
Animated satellite photography depicts current 
movement as seen from space (currents move in one 
direction during El Niflo and in the opposite direction 
under normal conditions). These step-still satellite 
sequences of the 1983 El Niflo were obtained from the 
California Space Institute, a research unit of the 
University of California, located on the Scripps 
campus. Additional remote sensing images indicate the 
locations of phytcplankton populations (a favorite food 
of the tuna) in drder to illusu:ate some of the techniques 
fisherman use to locale large populations of tuna. 

Exhibit 4: Sharkey's Believe It or NoL 
Unlike the other prototypes, Sharkey s Believe It Or 
Not was designed for sustained interaction as part of the 
aquarium*museum*s cL sroom and interpreter-guided 
activities. Sharlcey*s defuses the sensationalism and 
myths surrounding sharks while using "gee-whiz" 
statements to capture the attention of young children 
and adults alike. Don Wilkie, Director of Scripps 
Aquarium-Museum, dissects a shark and shows viewers 
the sharic's gut and other internal organs. 

The program begins with a continuous attract loop in 
which "believe it or not" statements superimposed over 
footage of swimming sharks shown on a NTSC 
monitor. A menu of these "believe it oi not** statements 
is displayed concurrently on a Macintosh screen and is 
available to the viewer at many key points in the 
program, thus encouraging pursuit of individual 
interests. 

A submenu lists additional choices: "Jaws and Teeth," 
"Senses," "What Sharks Eat»" and "Digestion." Viewers 
who choose to learn about what sharics eat, for example, 
see still frames of various sharks, each accompanied by 
a description of its eating habits. A video dissection of 
the shait ' ^ uZ\ valve (part of the digestive system 



with functions similar to the stomach/intestine in 
mammals) is also accessible. This portion of the 
prototype serves as an example of how multimedia 
technology can ease resource burdens by substituting 
electronic demonstrations for dissection. 

Use of the Prototype 

for Planning and Fundralsing 

Shortly after the prototype exhibits were completed, 
Scripps Aquanum-Muscum hired an exhibit curator, one 
with considerable interest in developing interactive 
multimedia-based exhibits. As of August 1990, the 
exhibit curator, working with a team that included one 
of the (former) graduate student project managers, 
completed conceptual plans for the 3,700 square foot 
museum portion of the new facility. The new museum 
is built around the theme "Er^ploring The Blue Planet," 
and provides visitors with an overview of ocean 
sciences. The first exhibit area presents a history of 
oceanography, from the earliest "explorers" (ancient 
civilizations), to the start of modem oceanography in 
the early 1870's, to the present day. Visitors are invited 
to become oceanographers and explore the sea. Exhibits 
are based upon the theme of water itself (chemistry and 
movement), water with air (climate and weather), water 
with land (plate tectonics and sediments) and water with 
life (marine biology). A final series of exhibits invites 
visitors to consider the future of the Earth and 
emphasizes occanographic reseaxh as it investigates 
global ciiange. The exhibits en^^phasize exploration, 
hands-on experiences, and the scientific process. 

Three interactive videodisc exhibits are currently planned 
for the new facility. Though Scripps staff was pleased 
with all four exhibit prototypes, only one will be 
included in the first phase of exhibit development and 
construction. That exhibit. Exploring an Oceanographic 
Research Vessel will be appear as part of an exhibit on 
the history of oceanography. This portion of the exhibit 
illustrates oceanographic instrumentation, including the 
vessels used for ocean exploration. Future plans call for 
additional interactive multimedia exhibits including 
several that are stand-alone interactive computer 
exhibits. 

As a resuh of the project, the Aquarium-Museum has 
expanded its range of exhibit formats, planning more 
complex and more sophisticated exhibits that build on 
the student prototypes. By the Fall of 1990, Scripps 
will have produced a promotional videotape that 
explains the exhibits to potK'.nUal sponsors. Narrated by 
former astronaut Wally Sch.'rra, the tape will illustrate 
how the interactive exhibits work. Though the exhibits 
themselves are videodisc-based, a tape lends itself to 
more convenient demonstrations for interested parties. 
The viewer will see actual footage of aquarium visitors 
interacting with the prototype exhibits as well as 
additional examples of the types of video footage that 
will be incorporated into the final products. 



ERLC 



217 

287 



Overview of Exhibit 

Prototype 



Many years ago, a French ship named Calypso captured the imagination of millions of television 
viewers. People worldwide were invited to explore the ocean— how it moves and breathes; its 
dramas and seasons; how it feeds a host of living things; what harms it and what nourishes it. What 
the viewing audience may not have known however, was that Scripps institution of 
Oceanography (SlO) was uncovering secrets of the ocean long before the Calypso. 

Exploring an Oceanographic Research \/esse/ informs aquarium/muse, visitors of the role Scripps 
vessels play in the critical research of the mother of waters e ocean. We hope that the viewer will 
come away with the idea that the ocean is no longer a dark and sinister abode of secrets and 
terrors. We also want the viewer to realize at resea ch is nrx>re than merely the compilation of lists. 
Rather, through our understanding of the ocean, we al! share in the responsibility for its future. 

General Description of the Profect 

Exploring an Oceanographic Research Vessel is an educatfonal project undertaken by the 
students and staff of the Educattona! Technology masters degree program at San Diego State 
University in conjunction with Scripps Institution of Oceanography (SlO). This endeavor is 
directed toward the development of an interactive vkJeodisc program designed to denwnstrate 
essential aspects of a research vessel in a museum environment. 

Scripps operates throe basic types of research vessels. The vesse' chosen for devetopmerrt in 
this exhibit prototype is the RA/ New Honzon, a typical surface ship. The conrpleted exhibit is 
envisioned to also include the R/V FLIP, an acronym for floating instrument platform, and the fl/V 
ORB. an acronym for oceanographic research bouy. The vessels were chosen to represent three 
different but basic types, sizes, shapes, and specific uses of research vessels. 

This Is one segnrrent of a larger da ;s project. Other teams in the interactive video class developed 
segments on: El Niffo; Fish; and S'.iark Dissection, for SlO. Teams consisted of six members 
possessing various degrees of expertise in the assigned development areas. The duration of 
commitment on this project was eleven weeks. 

The computer software used in this project was designed especially for use with interr 
videodiscs and is described In more detail in another secttor of the documentation. Compu\c 
graphics were used for: conveying informational messages, tnies. and fcons. All computer 
graphics were developed by team members. 



SDSU Department of Educational rechnoloiy 
Prototype Interactive Video Exhibits 

Figure L Excerpt from narrative portion of design docurr ntation developed for Exploring an Oceanographic 
Research Vessel 
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Shore 
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See another 
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Let me explore 
on my Own 



Tell me about 
Scripps* Fleet 
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Schematic 
of Ship 
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Tour of 
Main Lab 
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Tour of I 
Ocean Lab j 



See what 
happens here 



I 



See more about 
Equipment 



NO 



See other ships 



See another area 
of New Horizon 



YES 



Figure 2. Portion of a block diagram for Exploring an Oceanographic Research Vessel. Diagram shows 
interrelationship between major program components. 
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one-celled and are the first Unk In the ocean food 
chain. 



Animal plankton, called ZOOPLANKTON. have more 
than one cctt and eat the plant plankton or other 
zooplankton. 
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Figure 3. Sample storyboard specifications for menu from Exploring an Oceanographic Research Vessel Band of 
buttons at the lop of proposed HyperCard display is a "main menu" (available at numerous locations throughout the 
program) which indicates where viewer is located. Abbreviations: VF = video freeze, VM = video motion, CT = 
computer text, CG = computer graphics. Mo = motivator, EG/nik = expository generality/nice-to-know. 
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Figure 4* Sample storyboard specifications for a menu from Exploring an Oceanographic Research Vessel. leg/di = 
inquisitcsry instance/directions. 
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Figure 5. Excerpt from flowchart detail for Exploring an Oceanographic Research Vessel. 

282 

2.92 



Interactive MuItiMedia Museum Exhibits 




Figore 6. Sample HyperCard screens from Exploring an Oceanographic Research Vessel Labeled buttons permit 
visitor to select location for '"toui^ via video segment. 
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Equipment is launched and recovered from the after deck. 

LET ME LAUNCH 



"TTZri usad to measure the temperature of ocean water 
I (expendible bathythermograph) 
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Used to collect samples of ocean plants and 
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Nansen 
Bottle 




CLICK ON THE ECXJIPMENT YOU WAMTTOUSE 



There are many types of nets. They have different 
shapes, different sizes, and capture different plants 
and animals. Scientists select a net depending on what 
they want to coliecL 




Mld«water 
neis 



Surface-water nets 



Bottom-water nets 



Figure 7. Sample HyperCard screens from Exploring an Oceanographic Research Vessel Menu permits visitor to 
select specimen collecting devices. Once selected, viewer sees video segment showing how crew launches and 
operates device. 
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Abstract 

The user interface is an important component of an 
interactive system because it represents the totality of 
communications between the usei and the system. The 
use of input devices and screen design arc key areas of 
consideration in the building of an interactive system. 
The purpose of this paper is to give guidelines for 
system developers in these two areas. 

introduction 

Advances in technology has given new importance to 
the user interface. As more and more people have 
become users of interactive systems, systems developers 
have found that the use ot input devices and screen 
design must be more carefully considered than ever 
before. A wide variety of input devices, including the 
keyboard (both data and cursor control keys), mouse, 
touch-sensitive screen, light pen and voice among 
others, that may be used. There are also a number of 
screen design formats, such as question and answer, 
selection, hierarchical menu (horizontal choices), pull- 
down menu (vertical list), icons and windows. What arc 
the key considerations for the use of input devices and 
screen design formats? In the paper we will discuss 
some guidehnes that should be used by interactive 
system devdopers. 

Guidelines for the use of Input Devices 

In this section we will look at guidelines for 
implementing interaction through screen displays. 
Interaction between the user an the system is sometimes 
called dialogue, similar to a conversation between two 
individuals. Besides considering rules for implementing 
effective dialogue, we will explore general rules for 
using different input devices. 

1. Minimize the number of key strokes. To 
perfmn tasks and move between menus, the number of 
keystrokes, or other required actions, to complete the 
functions .should be as few as possible. If users are 
working with codes, they should be short in length. The 
depth of menus should be taken into consideration at 
design time. 

2. Use a two-level command or menu 
selection structure. If the system is driven by 
commands, or a menu, then the instructions should 
exist at two levels, one for the beginner and one for the 
more experienced user. A beginning user does not mind 
selecting "^D" for document, "P" for print, and "Go" to 
start the process of printing. As the user gains 
experience, this three stq) process, is abbreviated to 
entering DPG (in one stq)). The same principle should 
be applied to menu selection when the user is very 



familiar with the system. This two-level command or 
menu selection structure is accommodating to users 
with different levels of expertise in using a system. 

3. Help the user navigate through the system. 

Effective dialogue and functionality are fostered when 
the user understands how to progress thiough a system. 
This concept is analogous to reading the symbols on a 
Toadmq). To determine which way to go, the 
symbols (or prompts) must be understood. Likewise, in 
a computer or interactive system, the user must 
recognize whether they are in the mode to look at 
production rules, change a rule, or observe the execution 
of a set of rules. They also must understand how to 
move from one mode to another. Proper symbols and 
understanding of how to navigate through a system a>^ 
important for the user. 

4. Provide on-line help. If the user is unsure of 
backward chaining, he or she should be able to invoke 
the help system. If the user d'^ not remember what 
information is to be entered, then an on-line help option 
could prompt the user. On-line help is supplementary to 
the formal training for and written documentation of the 
system. These help systems are considered standard 
features, not only to interactive systems, but to any 
computer information system. 

5. Quick response time should be guaranteed. 

Response time is the amount of time (in seconds or 
smaller time units) that it takes for the computer 
system to respond to the user input. Users on 
mainframe or microcomputer systems are accustomed to 
a short (1 to 3 second) response time. No one likes to 
wait to get the task completed. This is particularly 
important in multiuser or networked systems. The 
response time is a function of both the hardware and the 
software. Response time is important at design time and 
also after the system is implemented because of 
continual changes in the system and working 
environment. 

6. Avoid blank screens during any dialogue 

session. As a response in a dialogue session, the user 
should see an £^propriate response. Many times, there 
are searching and sorting techniques that, unfortunately, 
take considerable time (or so it seems). Instead of 
leaving the display screen blank, some message 
appropriate to the task should appear to the user. For 
example, if the interactive system, is firing a rule base, 
then a message such as "Searching the rule base" should 
apoear on the screen. Users are likely to be more patient 
if there is an explanation for the l^q^se of time. Hence, 
for the user, appropriate messages arc preferred to blank 
screens. 
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7. I^aplement an undo feature in the 
commands. An undo comnnand is one that reverses a 
command or action that has just been implemented: 
However* not all actions in a system are reversible or 
undoable. Users have become accustomed to this helpful 
system interface, making the implementation of the 
feature by programmers a useful design enhancement 

8. Simplify error messages.Error messages, 
warning messages, and other related feedback should be 
in terms that are understood by the user. They are 
information to the user, not the programmer. For 
example, a message, "Check disk drive door to make 
sure it is closed," is a better feedback to the user than 
"System failure: abort, retry?" Messages like "syntax 
error" or "Runtime errw 256" are not particularly useful 
to users. In addition, users should be involved in 
approving the messages that are built into the operation 
of an interactive system, to ensure their understanding 
of them. 

9. Provide for atypical users. Most computer 
systems are designed for pe<^le who have 20/20 vision, 
are right handed, and have full-color vision. Many 
people, however, have vision problems, are color-blind, 
or are left handed Input devises such as numeric keypad, 
or trackball, may be difficult for left-handed users. Color 
display screens, for color-blind users, should appear 
readable and discernible on black and white monitors. 
An interactive system that is to be used by a wide 
variety of users must take into consideration the 
atypical user. 

10. Minimize the number of changes. If the 
keyboard system is combined with another device such 
as a mouse or light pen, minimize the changes from 
keyboard to the other devices. When another input 
device is used together with the keyboard, frequent 
changes between the two styles can be confusing. 

Guidelines for Screen Display 

Some general guidelines or directions for effective 
screen display in interactive systems are described as 
follows. 

1. Use uncluttered, well*spaced screens. The 
user screens should be uncluttered with appropriate 
spacing between fields, lines, and text Experts advise 
that only 15 to 20 percent of the screen space should be 
utilized Unnecessary items are distracting to the task at 
hand 

2. Avoid paragraphs of text at the screen. 

Paragraphs of text should rarely be used on the screen. 
Reading from a display terminal is not the same as 
reading from or browsing through a book, so organize 
information into vertical lists instead of paragraphs. If 
paragraphs are necessary, however, display pan of the 
text^ then allow the user to press a key to continue with 
the next part A mixture of upper-case and lower-case 
text is more readable than using all capitals. 



3. In general) left-justify text and right- 
justify numbers. Text and number lists should be 
distinctly formatted. Lists of text should be left 
justified; that is, the leftmost letter of each word in the 
list should be aligned vertically to the left. In contrast, 
numbers should be right justified, aligned vertically to 
the right. When using decimal values with varying 
decimal places, the list of numbers should be decimal 
aligned. 

These examples are shown below: 
Left Justify Right Justify Decimal Align 

Investments 4,300 34.78 

Slocks 543 123.6 

Bonds 678,413 6.789 

This arrangement of lists is easiest for the user to read. 

4. Limit the number of menu selections from 
five to seven. In general, a user c;m retain 
information from about seven items «t a time. 
Therefore, the number of selections frOm a single menu 
or pull*down submenu should be limited to between 
five and seven items. Each one should be numbered 
(1,2,3 etc.). Use letters only with a short menu list of 
three or four selections identified by codes (i.e., D for 
delete, U for update, or A for add). The items can be 
listed by time, function, number, v iportance, or other 
criteria. A general rule of thumb, however, is to list the 
menu options by frequency of use, with the most 
frequently used at the top or left of the menu screen 
(breadth) and by the highest level of menu (depth). 

5. Be consistent in wording and location of 
prompts. The wording and location of prompts, fields, 
and other identifiers should be consistent throughout the 
screens. Use uniform wording of phrases. For example, 
do not use change the record on one screen and update 
the record on another. With date entry fields, maintain 
the same formats for name, social security number, 
date, and so on throughout all screens. The screen 
formats should match the formats used in printed 
reports. The prompts to the user should always appear 
in a corresponding location. If the lower-left hand comer 
of the screen is used to display the message on how to 
move to the next screen, retain that relative positioning 
in all appropriate screens. Keep the language consistent 
with the expectations of the user of the system. This 
means to adhere to the vocabulary of the user. If the 
user is an accountant, then the term debit and credit are 
fine, but they arc not appropriate for the general user 
referring to money paid in and money paid out. 

6. Use reversed videos or color for data entry 
fields. If a user is filling in the various items on a 
form-like screen, each data entry field should be 
highlighted. Reversed video (the opposite of the screen 
bapkground: black letters on a white background or 
white letters on black background) is one popular 
method in many business-oriented information systems. 
Color is another way to draw attention to a particular 
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field. Windowing is still another technique used to 
highlight the fields to be used. The system should not 
permit the user to alter information or words that are 
not to be changed on the screen. In other words, the 
programmer should "protect" these screens when the 
screen programs are being implemented. 

7. Use color and graphics to add to screen 
design. Color can be a useful asset to screen design if 
it is available on the video system. It can be used to 
highlight; 10 readable. Color on diagrams makes 
intricate detail more discernible for the user. Caie should 
be taken in selecting the color of the background (blank 
screen) and the color of the text that appears on the 
screen. Certain color combinations are hard to read, such 
as red letters on a blue background, for example. The 
designer of the system should make sure that if the 
color screens are displayed on a black and white 
monitor, that the visual effects of the screen are still 
discemible and pleasing to the eye. 

8. Let the user select some characteristics of 
the screen interface. For those applications where 
usage is extensive by the same user, some method of 
configuring or tailoring the screen environment for the 
individual should be available. For example, if color is 
used, then the user should be able to select certain 
colors for his or her working environment If a beep or 
sound is used to identify the end of a field or other 
warning, then the user should be able to turn the noise 
or sound off. Different people have different reactions to 
color, sound, and so on, and these minor characteristics 
can be very annoying to those who use the system 
continually. If the users, on the other hand, vary from 
day to day, then the screen characteristics should be 
determined by the designer of the system. Advising 
systems used by the general public are a type of 



interactive system where each user should not be 
allowed to change the characteristics of the terminal 
display. 

The preceding rules are to be considered as guidelines for 
general screen design for interactive systems. The more 
specific and narrow the domain of the interactive 
system, the more these rules may have to be adjusted to 
fit the functional needs of the specific interactive system 
application 

These guidelines are to be considered during the design 
stage of the interactive system life cycle. Incorporating 
user-friendly procedures is easiest and most effective 
during the aesign stage. The best judge of a user 
interface is not a given set of rules, but an evaluation 
by the user. Attention to the user interface design will 
reap countless benefits to the acceptability, 
functionality, and efficiency of the intendeu interactive 
system. 
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Abstract 

Methods of adding research tools to computer-based 
yndeo instruction (CBVI) programs are discussed. IBNTs 
InfoWindow Presentation System (IWPS) authoring 
system, with IBM's InfoWindow® Touch Display, is 
used as an example of one CBVI environment. 
Segments will be used to illustrate the addition of 
research tools to a CBVI program, for example, learners' 
demographic data gathered by the author-developed 
Subject Information Form (SIF) tool. 

Introduction 

Research and development techniques are used to build 
knowledge and understanding of how different teaching 
media and methods affecf learning (Laurillard, 1987, p. 
74). In pedagogical designs for computer-based video 
instruction (CBVI), therefore, accompanying research 
tools are often planned. These are usually paper-and- 
pencil. Today, I will discuss methods of adding 
computer-based research tools to CBVI programs. I used 
CBVI for my dissertation research on learner control 
options (Hassett, 1989, Nov. 13-15; Kassett, 1991, 
anticipated). Originally, paper-and-pencil tools were 
planned, with the exception of the Embedded Content 
Questions tool (CBVI items). After designing the CBVI 
program and entering the authoring stage, I determined 
to add other research tools to the instructional design. I 
will use this experience as an example and share about 
these tools. All but one of the planr cd paper-and-pencil 
tools were adapted to become computer-based tools. 

Revlaw of Literature 

The literature on adding research tools to CBVI 
programs is limited Three articles lend some support to 
the subject Aiken and Mikan summarized selected 
items on computer use in 550 undergraduate nursing 
programs (United States). These were results of a 
national survey conducted in 1987 and rq)eated in 1988. 
Findings on nursing program administrators who 
responded "Yes" to non-instructional uses of computers 
Qi » S09, in 1988) included: S3 percent for calculation 
(grades), 39 percent reports, 25 percent data 
management, and 24 percent for data analysis (1989, p. 
106). 

Boyce and Rieber (1989) compared student responses to 
a paper-and-pencil faculty evaluation «ool versus a 
computer-based tool having the same items. The 
computer-based tool was not added to a CBVI program. 
However, findings lend support to adapting existing 
paper-and-pencil tools to computer-based ones without 
significant differences in student responses. There were 



no signiflcant differences in time taken to complete 
either tool. And few si;jnificant diflerences were found 
between student responses for each item; for example, 
students using the computer-based tool included more 
open-ended comments, 

Rizzolo's three-round Delphi study (1990) identified 
factors that experts believe impede the developnient and 
use CBVI in nursing education. Nurse educators with 
experience in CBVI development were the identified 
experts {N = 31). Results included one of the essential 
features of exemplary CBVI programs as: record keq)ing 
for research on CBVI programs (Af = 4.29 [agreement 
received ratings of 4 or 5], = .74, rank = 12 [out of 
22]; p. 156). 

Methods 

Methods of adding research tools to CBVI prc)grams are 
these: (a) add before the main program begins; (b) add 
after the main program ends; (c) add to the main 
program itself (if one is the author); and (d) a 
combination (e.g., add a tool before and another after the 
main program). The added tools may be run during the 
program by adding them to the authoring phase. This is 
what I did; however, the added tools do not have to be 
incorporated into the main program. They may also be 
run separately, prior to or after the program (for 
example, using a computer language program such as 
one written in C). The added research tools may also be 
nm by attaching them to the existing program (such as, 
by using a DOS batch file). Data handling should allow 
the researcher automated access to subjects* data. 

If a desired paper-and-pencil tool is protected by 
copyright, then permission must be obtained bcfoie 
adapting it to a computer-based tool (Copyright basics, 
1989; Copyright law, 1989; International, 1989). If a 
computer-based tool is available to add, and is protected 
by copyright, then permission must be obtained before 
adding the computer-based tool to a CBVI program 
(Hoyden, 1984; Copyright registration, 1986). 

Example 

I will show segments of added research tools today as an 
example of one CBVI environment. I used IBM*s 
InfoWindow Presentation System (IWPS) authoring 
system (1986) with an IBM InfoWindow® Touch 
Display for the CBVI program with research tools 
added. An IBM PS/2® Model 70 microcomputer was 
used, with DOS 4.01. Tlie IBM InfoWindow System is 
presented as an example of one CBVI environment 
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Figure 1. Manual and Compuicr-Bascd Research Tools. 

Other authoring systems could be used to run computer- 
based research tools, or a programming language could 
be employed, such as Assembly, BASIC or C. Major 
strengths of the InfoWindow System include (a) graphic 
overlay and control for both touch-screen and voice 
synthesis functions on an intelligent terminal, (b) 
random access to videodisc segments (Lambert, 1987, p. 
99; Hannafin & Peck, 1988, pp. 357-358), (c) the 
ability to run video images and graphics on *c same 
display, and (d) capability for user exit calls to C 
language programs or code modules. 

IBM's IWPS authoring software (Level 051 (A)) was 
used to adapt the research tools (1989, Aug. 3). Three 
tools were added to the CBVI program within a 
reasonable period of time (3 weeks) by using a rapid 



prototyping approach (Hassett, 1990). Two of these 
tools were author-developed and the other is public 
domain. I was surprised at how much graphics creation 
was necessary in order tc add the tools. 

The text for each added tool had been word processed 
ahead of time because of the intention to print papcr- 
and-pencil versions. I imported the word processed tools 
directly into IWPS by: (a) isolating each item to a 
single "screen" in my word processor (I changed the 
background and character colors to vary the screens, and 
stripped off the WordStar v. 5.5 menu bar, etc.). (b) 
capturing each isolated screen with the Picture. Taker 
function of IBM's Sioryboard Plus Version 2.00 
graphics package (IBM Storyboard, 1989, June, Ch. 3), 
(c) using the IWPS Process Graphics option to import 
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Figure 2. Subject InforroaUou Fom: Sample Computer-based Screen with Author-defined Touch-Sensitive 
Keypal 



SCTeens from St(»yboard (IWPS Editor. 1986. p. 7-1). 
and (d) adding graphics panels to the imported screens, 
for buttons, etc.. by importing them from Storyboard as 
overlays (IWPS Editor. 1986. p. 7-1 1). Sec Figure 2 for 
a sanq>le screen that combined captured word processed 
text with a gn^)hk overlay: an auth(»^-defined touch 
sensitive keyboard (IWPS Editor. 1986. p. 9-79). 

I will now discuss die method of adding research tools 
to my program. I used the combination method^ that is. 
I added two tools before and another after the CBVI 
program (incorporating them into the program). Figure 
1 depicts data collection for subjects by research 
tool(The ECQ was planned as computer-based (CBVI) 
from the outset; its boxes are shaded on Hgure 1.) 

Add before the Program Begins. The author- 
developed Subject Information Form (SIF) tool 
(Hasseu. 1989c) was added to collect five areas of 
learner demogr2q>hic data* A sample cIF screen is found 
in Figure 2. The Screening Pretest (SP). also auihor- 
developed. was added to collect baseline subject 
knowledge immediately prior to the CBVI content 
modules (Hassett. 1989b). 

Add after the Program Ends. The Adjective 
Rating Scale (ARS) was added immediately after the 
CBVI program. Kelly. Pascarella, Terenzini, and 
Cteq>man developed the ARS, a public domain tool that 
measures attitudes toward courses and programs (1976). 



Add to the Program Itself. The author-developed 
Embedded Content Questions (ECQ) tool (Hassett. 
1989a) was already a part of the main CBVI program 
and did not need to be added. 

Data handling. The researcher needs automated access 
to the subjects* data. When I collected data for a Pitot 
Study and then dissertation study data, there was an easy 
method for accessing the individual subject data, here 
was no method for aggregating the data except 
manually. Since then. I had a program written in 
Microsoft C that places the data frtxn each subject in an 
aggregate table(Later. statistical programming will be 
aided.) 
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Abstract 

The purpose of this literature review was to examine the 
research concerning the effectiveness of instructional 
organizers when used in the design of instruction for 
Computer-Bascd Interactive Video (CBIV). Three 
techniques - the advance organizer, the structured 
overview form of graphic organizer and the pictorial 
graphic organizer - were considered. Two questions were 
asked. Do they work at all? Are they effective with 
CBIV? The evidence will be considered in light of the 
original theories, past and current research results. An 
alternate explanation will be advanced based on 
Witirock's Generative Learning Hypothesis and 
recommendations made for future research. 

Introduction 

The purpose of this paper is to review the research 
literature pertaining to the use of instructional 
organizers with a view to which, if any, v/ould be 
effective when applied to inr^truction using Computer- 
Based Interactive Video (CBJV). Instructional organizers 
are those given before, during and after, the lesson and 
include advance organizers and its offshoots, structured 
overview graphic organizers and pictorial graphic 
organizers. The review will first consider what evidence 
there is that any of these techniques have an effect on 
leammg (immediate recall) or retention. Do they woric 
ataU? 

Second, it will look at the relevant research on CBIV to 
determine if these techniques have been successfully 
applied to the medium. Even if they are useful 
elsewhere^ will instmclional organizers be effective 
when used in instruction presented by CBIV? Why 
CBIV? It is a medium which combines the power of the 
computer tc support student interaction and 
involvement with the richness of the various audio and 
visual media to offer the capability of producing multi- 
media student-centered learning environments. As such, 
it may jrovide a glimpse of the classroom of the future 
(Simonson and Thompson, t990). Determining which 
instructional design techniques are effective when used 
with CBIV could provide useful recommendations to 
those who wish to develop instruction using this 
powerful new medium. 

Instructional Organizers. All types of 
instructional organizers originated with the concept of 
advance organizer first proposed by David Ausubel 
(1960, 1963). While their differences lie both in the 
design and placement in the instructional sequence, the 
purpose of these organizer? are similar. They are 
instructional techniques designed to relate the material- 



to-be*leamed to the learner's existing cognitive strucuire 
to facilitate learning and retention. 

The Advance Organizer, The purpose of the 
advance organizer is to facilitate the learning and 
retention of meaningful verbal information. Ausubel's 
Subsumption Theory postulates that cognitive structure 
is hierarchically organized in terms of highly inclusive 
concepts under which are subsumed less iiiclusive 
subconcepts and informational data 

... [and, further] ... it is reasonable to suppose that 
new meaningful material becomes incorporated into 
cognitive structure in so far as it is subsumable 
under relevant existing concepts ... [Thus]... the 
availability of £q)propnate and stable subsumers 
should enhance the incorporability of such material 
(Ausubel, 1960, p.267). 

Advance organizers are introduced in advance of the 
learning material itself and are presented at a higher 
level of abstraction, generality and inclusiveness 
(Ausubel, 1963). Their main function is to bridge the 
gap between the learner's cognitive structure and the 
material-to-be-leamed and must be slated in terms 
familiar to the learner (Ausubel, Novak and Hanesian, 
1978). Ausubel proposed two varieties: 

1. the expository organizer: to be used when the 
learning material is unfamiliar and meant to 
provide an ideational umbrella under which to 
incorporate the new material. 

2. the comparative organizer: to be used to 
integrate new ideas with basically similar ones 
in cognitive structure by pointing out the 
similarities and differences between them. 

The Structured Overview Graphic Organizer. 

The second technique to be considered was first advanced 
as a "structural overview" by Barron (1969) as a 
modification of the advance organizer. In this approach, 
tree diagrams introducing the new vocabulary to be used 
in the material-to-be-learned were developed as a 
comprehension aid for low ability learners. It used the 
spatial characteristics of diagrams to indicate the 
relationships and distances between key terms (Hawk, 
McLeod and Jonassen, 1985). 

Barron (1969) believed that the stru<:tured overview 
assumed the properties of advance organizers because it 
was intended to relate new content information to 
relevant subsuming concepts in the learner's cognitive 
structure. Alvermann (1981), however, noted that 
graphic organizers were unlike advance organizers for 
two reasons: they were written at the same, rather than a 
higher or more abstract, level as the to-be-learned 
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material and they use lines, arrows and spatial 
arrangement to depict text structure and relationships 
among key vocabulary. While Ausubel does not exclude 
diagrammatic material, the first point would clearly 
distinguish structured overviews from advance 
organizers. 

While Barron (1969) first referred to the technique as the 
structured overview, he later accepted Ausubel's advice 
to rename the technique "graphic organizer" (e.g. Barron 
and Stone, 1974). Brown, Padron, and Rivera (1988) 
note that graphic organizers follow five general patterns: 
cause/effect, comparison/contrast, time/order, simple 
listing and problem/solution. 

The Pictorial Graphic Organizer. Hawk, 
McLeod and Jonassen have further developed Barron's 
modification. Their form of organizer is a more 
pictorial, visual, or graphic presentation than any of the 
organizers referred to earlier [advance organizers or the 
structured overview form of graphic organizer] (Hawk, 
1986). These pictorial graphic organizers take one of 
two forms: participatory organizers, in which students 
participate in the completion of the organizer, and final 
form organizers, in which they do not (Jonassen :jid 
Hawk, 1984). 

Scope of the Review 

This review is meant to be exploratory, to answer the 
question: Is there any evidence to justify the use of 
these techniques in CBIV instruction? With that in 
mind, a computerized search of ERIC and PsychLit was 
first carried out to s^ssess the current and recent state of 
the literature for each topic; that is, back to 
approximately 1975. Second, the references included 
with various articles located by the computer search 
provided an indication of the seminal articles for each 
topic. 

The literature on advance organizers dates back at least 
thirty years to David AusubeFs (1960) article recounting 
his study of 120 college students who received an 
expository advance organizer prior to learning material 
on metallurgy. That literature is extensive. Even ten 
years ago, Mayer (1979) noted in his review of the topic 
that hundreds of studies and dozens of reviews had been 
published since the 1960 paper. Since the purpose of 
this review is to gain an overview of the topic and 
discern how it applies to CBIV, the section pertaining 
to advance organizers will consider only published 
studies and will summarize several well referenced 
reviews on the topic. A caveat is that this approach may 
leave the review open to the effects of publication bias 
and lead to overly positive conclusions. There is, 
however, considerable variation in study design and 
methodological rigor within the studies included 

By contrast, the research on the structured overview 
graphic organizer and the pictorial graphic organizer 
appears to be much more limited. Consequently, all 
papers which could be obtained to this point in time. 
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including published studies, conference presentations, 
and textbook chapters, have been included. Similarly, 
the literature pertaining to CBIV is also sparse and all 
sources obtained have been included. 

Advance Organizers 

Do advance organizers work as Ausubel claimed? He, of 
course, provided evidence that it did. For example, one 
of his first studies, testing undergraduates on their 
learning from a 2500 word passage on metallurgy 
produced statistically significant results in favor of the 
group receiving the advance organizer (Ausubel, 1960). 
Many studies have examined the construct since and not 
all have been as supportive of the claim. Hartley and 
Davies (1976) reviewed advance organizers as a part of a 
broader overview of preinstructional strategics. They 
noted that the most favorable studies were the initial 
ones but that they were of limited generality. Some of 
the best controlled studies to that date produced miAcd 
results. In spite of the mixed evidence, they concluded 
that, in the majority of studies, advance organizers did 
facilitate both Iteming and retention. A major problem, 
they pointed out, lay in the design and writing of 
advance organizers because there were no {Hocedures nor 
operationally defined steps for generating them. 
Ausubel, though, vociferously disputed this on at least 
two occasions (Ausubel, 1978, Ausubel, Novak & 
Hanesian, 1978). He refers readers to his 1968 text on 
educational psychology. 

Barnes and Clawson (1975) were less generous. They 
reviewed 32 studies using a "voting technique" to 
organize them into groups according to whether the 
results were statistically significant or not. Non- 
significant results prevailed 20 to 12 leading the 
investigators to a negative conclusion. It should be 
noted that Barnes and Clawson did not separate the 
results according to the various types of organizers used. 
Thus, for example, the first study of "structured 
organizers" (Estes, Mills & Barron, 1969) was included 
even though these are not true advance organizers. They 
did, however, differentiate amongst the studies according 
to length of study, ability and subject type. In each 
case, the count favored non-significance. 

Mayer (1979a), refuted the Barnes and Clawson review, 
criticizing it on three points: that it was not sufficiently 
sensitive to Ausubel's theory, that it did not analyze the 
studies in sufficient detail and that it did not adequately 
discuss methodological problems in the studies. He then 
reinterpreted AusubeFs subsumption theory in terms of 
his own assimilation encoding theory and reported a 
series of nine studies supporting his contention. 
Assimilation encoding theory, a three-stage model of 
learning, posited that: 

1 . information is received into woridng memory, 

2. that anchoring knowledge must be available in 
long-term memory and 

3. that this anchoring knowledge must be 
transferred to working memory so that it can 
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be actively integrated with the received 
information. 

The theory predicts that the organizer will facilitate both 
the transfer of anchoring knowledge to working memory 
and its active integration with the received information. 
It also predicts that the advance organizer may have hq 
effect: if the content and instructional procedure already 
contains the needed prerequisite concepts, if the content 
and insu-uctional procedure are sufficiently well- 
structured to elicit the prerequisite concepts from the 
learner, or if the organizer does not encourage the learner 
to actively integrate the new information. Further, if the 
learner already possesses a rich set of relevant past 
experiences and knowledge and has develq)ed a strategy 
for using it, the advance organizer would not be 
effective (for example, a high ability learner). 

Thus, Mayer offered the following advice for 
constructing advance organizers: it should contain the 
following characteristics: 

1. "Short set of verbal or visual [my emphasis] 
information, 

2. Presented prior to learning a larger bod> of to- 
be-learned information, 

3. Containing no specific content from the to-be- 
leamed information, 

4. Providing a means of generating the logical 
relationships among the elements in the to-be- 
leamed information, 

5. Influencing the leaincr*s encoding process". 

In his own review of the advance organizer literature, 
Mayer (1979b) reiterated his position, noting that 
certain conditions must be met if advance organizers are 
to have an effect. The conditions are that the material 
must be unfamiliar to the learner and must be 
potentially meaningful. The advance organizer must 
provide or locate the meaningful context and must 
encourage the learner to use that context during 
learning. The learner must not already possess the 
relevant conceptual context for the material nor a 
strategy for using it (advance organizers should favor 
low ability learners). And finally, the test of learning 
should measure breadth of learning (transfer and long- 
term retention). 

Using only published studies which contained either an 
advance organizer group and a control group or a post 
organizer group, he developed a question-based voting 
technique for 27 studies. In the cases where there was no 
control group, post organizers could function in that 
capacity since they were given after the lesson and were 
therefore not a form of advance organizer. For each 
study, he attempted to answer the following three 
questions: (1) Is the material unfamiliar, technical or 
lacking a basic assimilative context? (2) Is the advance 
organizer likely to serve as an assimilative context? 
(3) Does the advance organizer perform better than the 
control group on a test? Mayer lists the results as 
percentages and for only three studies does he claim 
statistical significance. He concluded that when the 



advance organizer was used, there was usually a small 
but consistent advantage for the advance organizer 
group. He also concluded that advance organizers more 
strongly aided performance when material was poorly 
integrated, that they more strongly aided inexperienced 
learners and that they aided far transfer more than 
specific retention of details. 

Reviews of the advance organizer research use a variety 
of techniques from the traditional summary (Hartley & 
Davies, 1976), an either/or voting technique (Barnes & 
Clawson, 197S), to a question-based voting technique 
(Mayer, 1979b). Each of these is judgmental and not 
entirely convincing, although Mayer's review is based 
on theory. Two later reviews, however, use the more 
recent quantitative approach of meta-analysis. Of the 
meta-analytic techniques, the most cited is Glass* (e.g. 
Glass, McGaw and Smith, 1981) effect size statistic 
(E.S.) which allows the comparison of studies which 
vary in design, sample selection and setting in order to 
form conclusions. 

Tne first meta-analysis (see Table 1), by Luiten, Ames 
and Ackerson (1980), examined 135 studies yielding 
110 E.S. for learning (posttest within 24 hours of the 
treaunent) and SO E.S. for retention (posttest 24 hours 
and after). They calculated a mean E.S. of .21 for 
learning, which would indicate that the average subject 
receiving the advance organizer treatment would perform 
better than 58% of the control group individuals. The 
mean E.S. for retention was 26. Their conclusion was 
that advance organizers had a small, facilitative effect 
upon l2Qlh learning and retention. The effect on learning 
runs contrary to Mayer's predictions based on 
assimilation encoding theory as well as Ausubel's 
subsumption theory. Luiten et al also found advance 
organizers effective for all ability levels but especially 
for high ability learners (a mean E.S. of .23) which also 
contradicts both Ausubel and Mayer. 

The second meta-analysis (see Table 1), by Stone 
(1984), closely followed Mayer's model in order U. 
compare results with assimilation encoding theory. 
Consequently, she included only studies with a control 
groiip or a post organizer and with a posttest 
administered one week or later after the treaunent She 
listed 1 12 different E.S. from 29 studies. Stone reported 
an overall mean effect size of .66 which indicates that 
advance organizers facilitate the long term retention of 
new, unfamiliar material. However, she also compared 
"true" advance organizers, those which acted as 
subsumers (more general or abstract) to those advance 
organizers which were at the same level of generality or 
inclusiveness as the material-to-be-leamed. The mean 
E.S. for the former was .65 while the mean for the 
latter was .71. Stone also found lower E.S. for written 
and abstract organizers than for non- written (illustrative) 
and for concrete organizers. These results contradict a 
number of the assumptions of assimilation encoding 
theory indicated above and provide some support (or the 
use of graphic organizers (to be considered next). 
Finally, Stone found no special facilitation for low 
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ability or low knowledge learners, again contradicting 
assimilation encoding theory and subsumption theory. 

Table 1. A Comparison of the Luiten et al and Stone 

Meta-analyses 



Table 2. Corkhill et al. (Retrieval Context) 



Variable 


Luiten et a! 


Stone 


learning 


0.21 




retention 


0.26 


0.66 


low ability 


0.13 


0.26 


medium ability 


0.08 


0.64 


high ability 


0.23 


0.34 




Review 


Learning 


Retention 


Luiten etal 


110 E.S. 


112 E.S. 


Stone 




50 E.S. 



A comparison of several more recent studies (1984 - 
1990) of the use of advance organizers reflect the same 
inconsistent results. One study, reflecting different 
theory, will be considered separately. The others will be 
summarized by effect size (see Table 3) and briefly 
described. The first study (see Table 2) by Corkill et al 
(1988) consisted of six experiments to investigate 
retried context set theory. This theory holds that re- 
reading advance organizers before the post-test will aid 
retrieval of the information from long-term into 
working memory. Five used the same materials in a 
laboratory setting with undergraduate university students 
participating for credit Only the time of the post-test 
varied. The fourth experiment was conducted in a 
university labo^ory school with grade seven students 
using different materials. Their results produce an 
average effect size of 1 .47 for those receiving an advance 
organizer before the treatment but not prior to the test 
(no cue condition) and a mean E.S. of 2.54 for those 
rereading the advance organizer before the test (cue 
condition). 

The treatment for experiment 6 differed in several 
respects. Subjects were assigned to one of five groups: a 
"true organizer** (written according to Ausubel's 
guidelines) group which parsq)hrased the organizer (cue 
condition), true organizer/parq)hrase (no cue condition), 
true organizer/paraphrase flrst paragraph only (cue 
condition), a "false organizer" (unrelated nuuerial) group 
(cue condition) and a control group. As can be seen in 
Table 2, in all **true** advance organizer groups, a 
positive effect was demonstrated while the "false" 
organizer had a slightly negative effect. The results 
provide evidence to support the use of advance 
organizers both according to the theories of Ausubel and 
Mayer (encoding context) and for retrieval context 
theory. 

The other studies attempted to use advance organizers 
according to the prescriptions of Ausubel or Mayer. 
Thus, another study by Corichill, Bruning and Glover 
(1988) produced less conclusive results. 



Y n ^ r 1 m ^ n t 




No Cue 


Pxrtf 1 ^Oiwini7/*r/Pjiranhr?i<jp^ 


1.04 


0.71 


Pxnt 1 fOrffaniTfir/Kev Word^ 


0 57 


-0.13 


Pxnt 9 ^OrcraniTftr/Paranhnwp^ 


No 
control 




Expt.3 (After 1 week) 


2.56 


1.96 


ExpU (After 24 hours) 


4.93 


2.85 


Expt.5 (After 2 weeks) 


3.78 


1.91 


Expt6 (After 24 hours) 


3.87 


1.53 


Expt6 (Organizer/lst ParaKraph) 


1.45 




Expt6 (False Organizer) 


-0.11 




Mean 


2.54 


1.47 



In two experiments, they compared the relative effects 
of concrete (comparative - using an analogy) and 
abstract (expository) advance organizers on students' 
recall of prose. Both were laboratory studies carried out 
with undergraduate university students volunteering for 
credit The abstract organizer treatments produced a 
mean effect size of -.62 while the concrete organizer 
treaunents had a mean effect size of 2.2S. These results 
are somewhat conflicting in that they provide support 
for comparative but not expository advance organizers, 
yet both should be effective according to Ausubel and 
Mayer. 

A study by Lenz, Alley and Schumaker (1986) 
investigated the effects of a regular teacher's delivery of 
an advance organizer prior to each lesson on Learning 
Disabled (LD) students' retention and expression of 
information from a given lesson. Teachers were trained 
to write advance organizers and the students to make 
notes on them. Student learning was assessed by an 
after-class interview in which was recorded the number 
of statements by the student related to the lesson in 
which the organizer was used. Results indicated 
improvement both after teacher training and again after 
student training. The Hrst improvement can be 
attributed to the use of the advance organizer per se and 
conflicts with Mayer's theory which indicates that it 
should not be effective for immediate recall. The student 
training result could be ascribed as much to the 
generative activity of note-taking as to the advance 
organizer. 

IGoster and Winne (1989) attempted to define and write 
advance organizers based upon assimilation encoding 
theory. Thus, a "concept" (expository) organizer was 
written at a higher and move inclusive level than the 
material-to-be-leamed and an "analogy" (comparative) 
organizer was congruent to (but different than) the new 
material. This study was conducted with 227 students 
from all eighth grade mathematics classes in a regular 
high school. 

A mean E.S. of -.13 was obtained for the concept 
advance organizer and 0.06 for abstract advance 
organizer. Based on effect size calculations, this 
indicates a slightly negative effect .for ad* ice 
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organizers. TKe investigators did, however, note some 
evidence using trace analysis to indicate that advance 
organizers increased achievement slightly. 



Table 3. Recent Advance Organizer Studies 



Study 


Learning 


Retention 


Corkhill Bruning & 
Glover (1988) 






Comparative Organizer 


-0.05 




ExDOSitory Organizer 


-0.27 


— 








Lenz, Alley & Schumaker 
(2986) 






Students 1-4 (Teach, 
training) 


l.ry9 


— 


Students M (Stud, 
training) 


3 49 


— 


Students 5-7 (Teach, 
training) 


0.97 


— 


Students 5-7 (Stud, 
training) 


2.36 


— 








Kloster A Winne (1989) 






Comoarative Organizer 


— 


-0.13 


Exix)sitory Organizer 


— 


0.06 








Gilles (1984) 


0.015 


0.33 








Doyle (1986) 


0.74 


1.03 








Tajika, Taniguchi, 
Yamamoto & Mayer 
(1988) 






Fragmented Pictorial 


0.078 


1.485 


Integrated Pictorial 


2.04 


4.08 


Mean 


0.426 


1.143 



Studies by Gilles (1984) with baccalaureate level 
medical surgical nursing content and Doyle (1986) with 
remedial college mathematics indicated advance 
organizers affected both learning and retention with 
stronger support for the latter, thus supporting 
assimilation encoding theory. Finally, a recent study 
(Tajika, Taniguchi, Yamamoto and Mayer, 1988) using 
a pictorial advance organizers with fifth grade Japanese 
mathematics produced similar, though stronger, results. 

Overall, based on the reviews and recent studies 
discussed^ the evidence does appear to support the 
contention that advance organizers have a facilitative 
effect upon learning and retention. This effect is clearly 
quite variable* however, reflecting the lack of clarity of 
what an advance organizer is and the difflculty in 
constructing one. What, then, of the other variations on 
the advance organizer concept? The next section extends 
this discussion to include two of its progeny: the 
structured overvie\^ form of advance organizer and the 
pictorial graphic organizer. 



Graphic Organizers 

Structured Overviews. As noted previously, the 
structured overview graphic organizer was originally 
developed as a modification of the advaiK:e organizer and 
was thought to assume those properties because it 
attempted to relate new information to relevant 
subsuming concepts. However, the structured overview 
differs from the advance organize in that it has a 
graphic form (a tree-structure diagram) and is written at 
the same level as the material-to-be-leamed. Given these 
differences, the question then arises whether this form of 
organizer affects learning or retentioii. 

A meta-analysis by Moore and Readance (1984) 
analyzed 23 graphic organizer studies with 161 
calculated effect sizes. They repented an average effect 
size of .22. Upon further analysis, however, they noted 
an average effect size of .57 for graphic post organizers 
and indicated that these tended to be constructed either 
by Hit instructor with the class or by the student They 
concluded that the structured overview form of graphic 
organizer does have an effect, especially for university 
students, that vocabulary learning is most positively 
affected and that post-organizers beneHt learners more 
than advance organizers. 

Again, more recent studies ai^)ear to support these 
conclusions (See Table 4). Alvermann (1981), in a 
study not included in the Mo<xt and Readance meta- 
analysis, found that partially complete advance graphic 
organizers had an effect on ninth grade students* 
comprehension and retention of text. The results 
indicated the strongest effect for )he less well organized 
text as predicted by assimilation encoding theory. Two 
studies by Boothby and Alvermann (Boothby & 
Alvermann, 1984, Alvermann & Boothby, 1986) found 
the graphic organizer to be effective as a strategy for 
facilitating fourth graders comprehension and retention 
of social studies text. While the second (1986) study did 
not test for retention, it did indicate that graphic 
organizers had an effect on a transfer of learning task - 
again predicted by Mayer's theory. 

Bean et al. (1986) reported a study in which tenth grade 
worid history students were divided into three groups: 
one provided with 14 weeks summarization training and 
taught to construct post graphic organizers, another 
taught the graphic organizer technique without 
summarization training and a third taught outlining 
instead. .A series of six multiple choice quizzes indicated 
a small effect for '^^mbineid organizer and summary 
training group but a negative effect for the organizer 
only group. The sixth quiz alone showed significantly 
higher scores /^r the group with summarization and 
graphic organizer training (only), perhaps showing 
evidence of a generative effect by the summarization 
training. 

Finally, Carr and Mazur-Stewart (1988) found that a 
vocabulary overview guide (a multi-page booklet) which 
included a graphic organizer was significantly superior 
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to a traditional form of instruction in improving the 
vocabulary comprehension and retention of community 
and technical college students. 

Table 4. Recent Structured Overview Studies 



Study 


Learning 


Retention 


Alvermann (1981) 






Descriptive Passage 


1.26 


1.76 


Comparative Passage 


0.06 


0 . 








Boothby & Alvermann 
(1984) 


0.98 


0.99 








Alvemumft. A, Boothby 
(1986) 






Passage 1 


-0.34 


— 


Passage 2 


0.45 




End of chapter test 


0.41 










Bean etal. (1986) 






Organizer and Summary 
Training 


0.16 




OrKanizerOnly 


-0.11 










Can and Mazur Stewart 
(1988) 


0.89 


1.23 








Mean 


0.42 


1.10 



While not extensive* the results of these studies are 
evidence that the structured overview for of graphic 
crganizer* eq)ecially the post organizer type, does affect 
learning. Given th^ readts» one must also ask if there 
is an effect for the pictorial form of graphic organizer. 
This question will be addressed next 

Pictorial Graphic Organizers. This organizer caii 
be described as moie pictorial* visual or graphic thai, 
other lands of organizers and has two forms: the fiP5u 
form (complete(9 graphic organizer and the paiticipatory 
fCMin (to be filled out by the learner during or after the 
lesson). 

A review of the relevant research by Hawk« McLeod and 
Jonassen (1985) and three more recent studies are 
reported in Table S. Two experiments by Jonassen and 
Hawk tested the use of teachor-constructed participatory 
pictorial graphic organizers in regular classrooms (grade 
eight history, grade twelve English literature). The 
results indicated a stronger effect for learning than for 
retention. A third study of eleventh and twelfth grade 
economics/sociology was also carried out but produced 
no significant I'esults nor was sufficient data available to 
calculate effect sizes. 

Two studies, by Hawk, McLeod and Jeane (1981) and 
Hawk and Jeanne (1983) were reported in Hawk, 
McLeod and Jonassen (198S). Statistically significant 
results indicated that participatory pictorial graphic 
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organizers facilitated both learning and retention. Again, 
insufficient data was available to calculate effect sizes. 
A more recent study by Hawk (1986) also found this 
technique to be effective in facilitating retention for 
above average students studying life science in the sixth 
and seventh grades. 

Darch, Gamine and Kameenui (1986) compared several 
techniques for informing six grade reading students of 
content information, including the cooperative (group) 
and individual completion of participatory pictorial 
gn^hic organizers, to a mcMfe traditional directed reading 
approach. The graphic organizers, especially for the 
cooperative learning approach, were found to be more 
effective in facilitating retention than the traditional 
dp^oach. Immediate recall was not tested 



Table 5. Pictorial Graphic Organizer Studies 



Study 


Learnlne 


Retention 


Jonassen ondHawk (1984) 






Experiment 1 


1.10 


0.63 


Experiment 2 


1.74 


0.73 








Hawk (1986) 




0.64 








DarcK Conine and 
Kameenui (1986) 






Cooperative Gnq)hic 
Ononizer 




1.59 


Individual (}re^)hk: 
Orsanizer 




0.72 








Alvemuvtn (1988) 






Self-percdved High 
Ability 


-0.64 




Self-perceived Low 
Ability 


3.94 










Mean 


1.54 


0.86 



Finally, Alvermann (1988) investigated the effects of a 
final form graphic organizer designed to induce tenth 
grade social studies students to look back in their texts 
for missed information. The organizer facilitated 
learning with low ability students but appeared to 
interfere with the learning of high ability students, an 
effect predicted by assimilation encoding ttieory. 

The evidence raited in these studies to support the use 
of pictorial graphic organizers, albeit meager, is 
consistently positive. Pictorial graphic organizers 
appear to facilitate the learning and retention of 
information, at least for younger learners. Immediate 
recall, though, appears to have been more consistently 
affected than longer-term retention of information. Like 
the two other organizer forms discussed, pictorial 
graphic organizers, in the participatory form, at least, do 
appear to be usttful 
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Organizers In Computer-Based 
interactive Video 

The second question to be answered by this review was 
whether or not instructional organizers effect learning 
v^hrn applied to instruction delivered by CBIV. The 
preceding discussion has suggested that, in general, 
there is modest, if conflicting, evidence that 
instructional organiz^^i^ do have some effect generally. 
While some of the studies reviewed did use visual and 
oral versions, the vast majority presented a text f(Mrm of 
advance organizers. Mayer's characteristics, however, 
indicated the acceptability of visual organizers, and so, 
it appears to be a reasonable leap of faith to presume 
that they might function in a computer-based 
instruction format and, particularly, computer-based 
interactive video, a technology which effectively 
combines both text and visuals. This, of course, 
presumes that the medium makes a difference. This 
continues to be a point of dispute among educational 
technologists. Clark and Sugrue (1988), for instance, 
point to decades of few robust differences in learning 
from different media. They liken media to delivery 
vehicles and claim that it is the method and not the 
medium that influences the psychological process that 
produce learning. Others take a different view. Hannafin 
(1989) notes a metamorphosis of computer*based 
instruction in the past decade which has provided 
educational technologists with **unparalleled tools for 
manipulating instructional strategies** (p. 167). He sees 
the increased potential for interaction as the fundamental 
diffeience between traditional and emerging technologies 
[such as CBIV]. While he agrees that the method is 
important, that educational technologists should be 
**concemed with the manner in which instruction fosters 
cognitive engagement...**, he claims that such 
engagement is mediated by a number of factors 
including the nature of the presentation stimuli (p.l70). 
The use of instructional organizers represent various 
possible methods of fostering cognitive engagement 
Will they do so when used in interactive instruction? 

Unfortunately, there ^pears to be little evidence as of 
yet to support the use of either advance or graphic 
organizers in a such environment, although Hawk, 
McLeod and Jonassen (1985) offer some advice for the 
design and preparation of pictorial graphic organizers (of 
cither type) for electronic text. Hannafin and his 
associates appear to have done the bulk of such research 
and have published a series of papers (e.g. Hannafin, 
1987, Hannafin and Hughes, 1986, Hannafin, Phillips, 
Riebcr, and Garhart, 1987) on **orienting activities** in 
CBIV based on the ROPES-*- (Retrieval, Orientation, 
Presentation, Encoding and Sequencing, **>** the 
influence of contextual factors) meta-model for 
designing instructional prescription (e.g. Hannafin and 
Riebcr, 1989). The **0*' in ROPES refers to **Orienting*' 
activities. An orienting activity acts as **a mediator 
through which new information is presented to the 
learner** (Hannafin and Hughes, 1986, pg.239). They 
include advance organizers (including the various 
graphic organizers) in their category of orienting 



activity indicating that they are **a variation of cognitive 
orienting ability*' (Hannafin, 1987, p.48). 

Hannafin and Hughes (1986) used both a video advance 
organizer and ^ final form graphic organizer in the 
development of a prototype CBIV lesson on rocketry for 
aerospace engineering stiidents but reported no data. In a 
study on the effect of orienting activities in computer- 
based instruction (CBI) which used upper class 
undergraduate and graduate university students, they 
found that both behavioral and cognitive orienting 
activities improved factual learning. Their description of 
the cognitive orienting activity was: **designed to 
provide an integrative method for establishing 
meaningful relationships, while also serving as a 
subsumer of lesson detail** (Hannafin, Phillips, Rieber 
and Garhart, 1987, p.80). This sounds like an advance 
organizer. However, the one example they give, **In the 
next section, you will be presented information abouL* 
The importance of studying cultures.** (p.77), seems 
insufficient according to Mayer's characteristics. 

Similarly, Hannafin, Phillips and Tripp (1986) used a 
one sentence cognitive orienting activity in a CBIV 
lesson on artists and art periods with 80 volunteer 
college students (graduate and undergraduate). They 
noted a significant interaction between the orienting 
activity and processing time but no main effect. 
Another study (Hannafin, 1987) compared the effects of 
orienting activities, cuing and practice on the learning 
of material on space voyages by ninth grade students. A 
significant interaction was found between the cognitive 
orienting activity and practice but the orienting activity 
alone was not demonstrated to be a significant 
instructional component Again, the example of the 
cognitive orienting activity provided, **Thc Next Section 
Presents the Following Concepts: ===> Unique 
lighting found throughout the solar system (and) 
Matter found throughout the solar system**, also seems 
meager. 

Rieber and Hannafin (1988) also studied the effects of 
textual and animated cognitive orienting activities on 
learning from CBI. Their subjects were 1 1 1 fourth, fifth 
and sixth grade students from a rural elementary school 
who were presented a lesson describing and explaining 
Newton's laws of motion. The orienting activities were 
of one minute duration each and consisted of either a 
text only, one sentence summary of the particular basic 
concq)t, an animated graphic sequence or a text and 
graphic sequence. There was also a control (no activity) 
group. The orienting activities were presented 
throughout the lesson before each basic concept. They 
found no statistically significant effects for any of the 
orienting activities. 

The orienting activities in these studies meet three of 
Mayer's characteristics for advance organizer 
construction: (1) a short set of verbal or visual 
information, (2) presented prior to the material-to-be- 
Icamed and (3) containing no specific content from that 
material. However, one could reasonably argue as well 
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that, for none of these studies, were the cognitive 
orienting activities of sufficient scope to meet the other 
two conditions; that is, (4) generate logical 
relationships among elements in the to-be*leamed 
material or (5) to sufficiently influence the learner's 
encoding process. The organizing activities used in 
these studies were not true advance organizers. 

One study which did present "true" advance organizers 
was carried out by Carves, Lindbeck and Griffin using a 
computer-based tutorial on kinematics with 100 
suburban high school physics students. The advance 
organizers were "written according to the gi'idelines of 
Ausubel et al (1978)" (p.785) and were presented on 
screen. There was no significant difference between the 
groups advance organizer treatment and a non organizer 
group (wh3ch read a related passage designed not to act 
as a subsumer). Krahn and Blanchaer (i986), however, 
tested the use of a "true" advance organizer to improve 
knowledge application by medical students in a 
computer-based simulation. Post-test scores showed a 
statistically significant difference between the 
experimental and control group, both for the total scores 
and particular questions designed to test far transfer, as 
predicted by assimilation encoding theory. The test, 
however, was given immediately after completion of the 
simulation and consisted of only six questions. No 
validity or reliability data were provided. 

Based on the results of these studies, there appears to be 
no convincing evidence yet to suggest that any of 
advance organizers, structured overview graphic 
organizers or pictorial graphic organizers would be 
effective if incorporated in instruction based on CBIV. 

Discussion 

Advance organizers were, of course, developed from 
Ausubers theories about meaningful verbal learning 
while Mayer analyzed the research in terms of 
assimilation encoding theory. Corkhill et al tested the 
consuuct of advance organizers in the context of 
Retrieval Theory. All were cognitive learning theories 
advanced to explain aspects of learning. Each theory 
posits that advance organizers would have a covert, 
internal effect on how the learner encoded the 
information presented. 

It is interesting to note, however, that pictorial graphic 
organizers and structural overview graphic organizers 
both appear to be most effective if presented in 
participatory form; that is, to be filled in or constructed 
by the learner. This may be the case. Hawk, McLeod 
and Jonassen (1985) point out, because "the more 
generative the nature of student partici^^atiori, the more 
likely it is that transfer and higher level learning will be 
affected" (p.l79). They are referring here to yet another 
cognitive learning theory witl different implications. 

This cognitive theory was developed by Wittrock (1974- 
who termed it the generativeSleaming hypothesis. 
Wittrock states that "it is the learner's interpretation of 
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and processin); of the stimuli, not their [the stimuli] 
nominal characteristics, which is primary" (p.88). In 
more simple terms, learners must construct their own 
meaning from teaching (Wittrock. 1985). Doctorow. 
Wittrock and Marks (1978). in a further commentary on 
generative processes, indicated that it involves not only 
generating meaning bat overt activities (generating 
associations among words, generating pictures, etc.) as 
well. 

In a comparison of Wittrock*s hypothesis to another 
cognitive theory. Rothkopfs concept of 
"mathemagenic" activities. Jonassen (1985) 
differentiated between the two orientations. One. which 
he labelled "constructive mathemagenic". accepts "that 
certain design attributes of any medium give rise to 
learning (because) the form or structure of the display 
inducer the necessary cognitive processes to produce 
learning" (p.9). The other, which he labelled 
"constructive generative", adheres to the belief that 
"what gets learned is more a function of the mental 
processes in which the learner engages then any 
attribute of the text display, because learning is a 
generative process - an act of constructing 
knowledge"(p.lO). 

There is some question (e.g. Grabowski. 1989. Stewart. 
1989) whether or not Jonassen is correctly interpreting 
Rothkopfs definition of mathemagenic behaviors. 
However, there is a distinction to be made between 
learning activities which require covert activity on the 
part of the learner and those which require an active 
response. While rejecting the label of mathemagenic. 
Stewart (1989). in his discussion of instructional text, 
accepts that there is a dichotomy. He labels the former 
"adjunct aids" instead. These adjunct aids include 
advance organizers, graphic organizers, adjunct questions 
a/id learning objectives. Jonassen (1985) lists several 
aids to text as "constructive mathemagenic". including 
adjunct questions, overviews and organizers. 
"Constructive generative" techniques, on the other hand, 
include such techniques as outlining, mapping and 
pattern-noting (p. 16). 

"True" advance organizers therefore are not generative. 
They elicit a covert response, rather than overt, active 
learning on the part of the learner. It is only the 
participatory form of the pictorial graphic organizer 
advanced by Hawk. McLeod and Jonassen and the 
student-constructed form of structured overview that are 
generative. Perhaps this explains the morass of 
equivocal results particularly from advance organizer 
research. Given the difficulty in determining exactly 
how to construct them, it may be that some 
investigators unkno'vingly designed their organizers 
such that the learners engaged in generative learning 
activities. Future investigation of instructional 
organizers might focus on the generative learning 
hypothesis, to determine if indeed the problem with 
advance organizers and the more static forms of graphic 
organizer is not a mauer of definition but of approach. 
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Recommendations for Future Research 
on Organizers In CBIV 

As has been seen, the research pertaining to the use of 
organizers, whether advance or graphh, in CBIV has not 
been extensive. Most has been carried out under the 
umbrella of the ROPES+ meia-model by Hannafin and 
associates. While they include advance organizers and 
graphic organizers in their model, it is doubtful thai the 
cognitive orienting activities they have used represented 
forms of these techniques. The lack of effect is not 
surprising, especially considering the modest results 
reported in past research. 

Based on this review, it is evident that three areas show 
much promise for future aicarch. These include: better 
desigiied organizers used with CBIV, interactive gr^>hic 
organizers and organizers based on generative learning. 
Each will discussed in turn. 

The first suggestion for research is that if advance 
organizers are to facilitate instruction presented via this 
medium, their design needs to be more thorough; that 
is, constructed according to Mayer's characteristics. As 
well, the medium of CBIV, with the potential for 
offering both computer graphics and video sequences, 
seems to be tailor - made for exploring the use of non - 
written advance organizers and the various styles of 
graphic organizers. Hawk, McLeod and Jonassen (1985) 
note, for instance, that pictorial graphic organizers could 
be provided in either print version or on screen. 

A second suggestion is that the research on screen 
graphics (e.g. Grabinger and Albers, 1988, Siliauskus, 
1986), on diagrams (c.g. Winn and Holiday, 1985) and 
on hypertext tools such as the Uarning Tool (Kozma, 
1989) and Notecards (McAleese, 1988) be applied to the 
development of effective electronic displays of pictorial 
graphic organizers with which the learner can interact; 
that is, fill in or modify. 

The final recommendation relates to the discussion on 
generative learning. It seems reasonable to suggest that 
research on generative learning may help fill in the 
gaps, to explain the limited and varied effectiveness of 
insuuctional organize. Some of the most impressive 
''esults were garnered vhen students were actively 
engaged in the learning process; for example, generating 
questions about the material being studied, constructing 
their own organizers or filling in graphic organizers. It 
may be that subsumption theory and assimilation 
encoding theory are indeed predictive of learning but 
only account for a limited depth of processing. Indeed, 
even Mayer (1984) reported evidence for the use of 
elaboration activities (behavior by the reader aimed at 
relating the presented material to existing knowledge) 
and concluded they help build external connections. 
Perhaps then, only when learning is generative, will 
students make effective use of techniques such as 
insu-uctional organizers to truly understand and 
remember. The power of CBIV, with its flexibility, its 
potential for interactive learning and its capability for 



decking the choices made by the learner, should be able 
to provide a strong contribution to the investigation of 
these questions. 

In summary, this paper sought to synthesize what 
research evidence reveals about the effect of instrtictional 
organizers, including advance organizers, structured 
overview graphic organizers and pictorial graphic 
organizers, on learning and retention. It further sought 
to determine what effect, if any, such techniques might 
have when applkxl to instruction delivered by computer- 
based interactive video, a promising, interactive 
medium. The results suggest that each technique may 
have a modest, if variable, effect across a broad range of 
learners and subject matter. Predictions from the models 
of Ausubel and Mayer were not consistently confirmed. 
Modifications to these models, or the conskleration of 
different theory, may be needed to account for these 
effects. 
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DIGITAL VIDEO INTERACTIVE: 
WHAT MULTIMEDIA DEVELOPERS THINK ABOUT IT 
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Abstract 

The Digital Video Interactive (DVI) Project at Virginia 
Tech, Blacksburg VA, is one of only a few university 
groups woridng in this exciting new multimedia field. 
To advance our knowledge of the technology, we 
conducted a survey of other DVI development sites. 
These sites were producing DVI-based applications 
ranging from point of information, to proof of concept, 
to commercid ventures. Our survey focused on five 
main topics pertinent to DVI application develc^ment, 
including what DVI-based application was being 
developed^ how it was done, problems encountered, 
development tools, and general impressions. This article 
documents responses of the interviewees as well as 
reports results common among interviews. It deals with 
questions of DVI application feasibility, cost 
effectiveness, and requirements through the experiences 
of persons knowledgeable in this area--DVI developers 
themselves. 

Introduction 

A research project between Virginia Tech and NCT, 
initiated in May 1989, has as a goal exploring the 
future of Digital Video Interaction (DVI) application 
development DVI™— Digital Video Interactive— is a 
new technology from Intel Cwpcnation for developing 
and presenting multimedia plications in a completely 
digital f(»inat Such multinniedia applications combine 
audio, still images, and motion video with text» 
graphics, and animation, and — most importantly — 
allow real-time manipulation of any \J these by a user. 
This real-time interaction is possible bec^*jse oil media 
are digitally captured and stored on a random access 
device (e.g., hard disk or CD-ROM). DVI can support 
72 minutes of full-screen, full-motion video and audio 
from a single compact disc. 

A DVI development system currently consists of a PRO 
750 Application Development Platform hosted on an 
IBM PC/AT-compatible computer with three special 
boards to support DVI: a video board containing the 
Intel video display processors; an audio board with a 
digital signal processor that provides multi-track, multi- 
channel digital audio; and a utility board with a CD- 
ROM interface. A DVI system also has an RGB 
monitor, an audio ampHHer and speakers, and a standard 
Sony CD-ROM drive, as well as add-on VRAM 
modules, video and audio digitizer modules, 40 
megabytes and large capacity magnetic discs, and run- 
time system software. Capture and playback of all 
multimedia components are supported by proprietary 
compression and real-time decompression techniques. 



Our project has developed some DVI applications, from 
which we have learned much about the current state of 
the technology, its strengths and weaknesses, and its 
potential for commercial success. However, in order to 
further advance our assessment of the technology, we 
conducted a survey of other DVI development sites 
during the summer and fall of 1989. Because of the 
newness of DVI technology, little was known at that 
time about the kinds of applications that were being 
developed on a DVI platform, the apfHopriateness of 
DVI as a development platform for these applications, 
the pros and cons of DVI development, and so on. 

Our survey focused on five main topics pertinent to 
DVI application develc^ment: 

1., what DVI-based application was being 
developed, 

2. how it was done, 

3. problems encountered, 

4. development tools, and 

5. general impressions. 

Most of the interviewees were application developers, 
but some were managers or project leaders. The kinds of 
projects being developed included point of information, 
proof of concq)t, and commercial ventures. 

The purpose of our survey was three-fold: 

to establish a communications link between 
our project and other DVI development sites; 
to learn first hand from the experiences of other 
development efforts, including the problems 
encountered and the methodology involved; aiKi 
to provide input for a multimedia taxonomy 
and a multimedia development tools 
requirements document that we are also 
producing. 

This article documents responses of the interviewees as 
well as reports results common among interviews. It 
deals with questions of DVI application feasibility, cost 
effectiveness, and requirements through the experiences 
of persons knowledgeable in this area — DVI developers 
themselves. 

Methods 

We developed a structured interview to be administered 
to DVI developers. All interviewees responded to the 
same set of questions. The interview was divided into 
two parts: 

• interviewee demographics — questions about 
background both in DVI and non-DVI 
technologies, and 
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• DVI-related information — questions related 
directly to each of the five topics listed above. 

Each interviewee was contacted to schedule an 
appoinunent for their interview, which took about an 
hour and a half to complete. Most interviews were 
conducted by phone, with the interviewer reading 
questions firom the interview form and recording answers 
as the interviewee responded. A few interviews were 
done in person and, in one case, the interview form was 
filled out directly by the interviewee. In all, ten 
interviews were completed, by persons at nine different 
organizations. The questions were aimed mainly at 
developers, but were appropriately rephrased to 
accommodate interviews with managerial personnel. 

The interviewee selection process was based upon 
availability and willingness to do the interview. 
Through our survey we wished not to collect 
information from any particular development site, but 
rather to collect information from any site with 



significant DVI experience. Many of the persons 
interviewed were either previou*ily known to us (e.g., 
from conferences, meetings, or other contacts) or were 
suggested through personal references. 

Results » » ' 

Interviewees^ Background Demographics. 
Table 1 displays inttrvieWee demographics. It SiKiws 
that the majority of those inMviewed were technical 
programmers or software engineers with approximately 
one year's experience with DVI-related work. Sevend 
interviewees were either managers within or presidents 
of their company. Two hW been involved with DVI 
since 1984, when RCA wns qiKOied the technology. 
Most interviewed had some ppor work with Interactive 
Video Disk (IVD) technology, indicating a natural 
transition to DVI work. Of the deve]ppers» some had a 
Bachelor of Science degreed computer science, some 
had advanced degrees, while others had only a minor in 
computer science. 



Organization 


Title 


Education 


DVI 

Experience 


DVI 

Training 


Related^ 
Experience 


Andersen 

Consulting, Dallas 
TX 


Senior 

Systems 

Analyst 


BS in 

Cjmputer 

Science 


18 months 


Intel 2 day 
workshop 


CD-ROM 


Applied Optical 
Media Corp., 
Malvern PA 


Pjresidcnt 


MBA 


18 months 


Intel 2 day 
workshop 


IVD 


CBIS, 
Atlanta GA 


Director of 

Product 

Develooment 


MS in 

Communicati 
ons 


9 months 


None 


None ^ 


General Dynamics, 
Groton CT 


Software 
Engineer 


BS m 

Computer 

Science 


18 months 


Intel 2 day 
worksh(^ 


CD-RQM,IVD 


Hcxizons 
Technology, Inc., 
San Diego C A 


Software 
Engineer 


MS in 
Geography, 
Minor in 
Computer 
Science 


3 months 


Intel 4 day 
workshop 


pn^hics, 
image* 
processing i 


Interactive Media 
Center, 
Ithaca NY 


Senior 
Programmer 


BS in Liberal 
Arts, Minor 
in Computer 
Science 


6 months 


Inlel 2 day 
workshop 


Graphics, I VD 


Software 
Engineering 
Institute, 
Pittsburgh PA 


Software 
Engineer 


MS in 

Computer 

Science 


12 months 


Intel 2 day 
workshop 


Graphics, IVD 


Virginia Tech, 
Blacksburg V A 


Graduate 
Student 


MS in 

Computer 

Science 


6 months 


Intel 2 day 
workshop 


Grapliics. , • 


Videodisc 
Publishing, Inc., 
New York NY 


Project 
Manager 


BAin 

Communicati 
ons 


since 1984 


No formal 


IV D 


Videodisc 
Publishing, Inc., 
New York NY 


President 


Unknown 


since 1984 


No formal 


IVD ^ . 



Table 1. Demographics of Interviewees 
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of the sites did not engage in much planning or design 
because they were not intending to produce a specific 
product, but rather were attempting only to learn more 
about DVI technology. 

The hardware and software used for development were 
fairly consistent among projects. Most were using the 
standard Intel PRO 750 personal computer with 
^propriate DVI boards and 4 Megabytes RAM, a hard 
drive (various capacity), Sony color monitor, mouse, 
MS-DOS, Microsoft C 5.1 compiler and linker, and 
Intel DVI run-time application software. Some sites 
were also using a Sony CD-ROM drive and extended 
memory system (EMS). Many of the sites originally 
started their development using a Dell System 310, but 
switched to the PRO 750 after Intel v^ianged hardware 
support 

3. Problems encountered. 

There were many problems encountered by all the DVI 
^plication development sites. The main problem that 
permeated through all tliC interviews was insufficient 
software documentation and product support. The 
development process at all sites was considerably 
hampered by inadequate documentation that lacked 
examples meaningful to basic DVI functionality. 



ORGANiZATION 


EXPERIMENTAL 


COMMERCIAL 


Andersen Consulting 




•Troccss Analyst Woricbench" 
Uses ELV to analyze efficiency for 
training employees 


Applied Optk:al Media 




•Truck Driver Safety" 
Training program for drivers to 
increase road safety 


CBIS 




Not an application; devetoping 
authoring tool for DVI possibly on 
a hij^h-speed iretworic 


General Dynamics 


"Submarine Surrogate Travel" 
Interactive navigation through interim 
of submarine 




Horizons Technok)gy 


"Aiioaft Video Textbook- 
Demonstrates mechanical work on an 
aircraft 




Interactive Media Center 


"Spanish Airplane" 
Interactive educational tool fo* 
smdents 




Software Engineering Institute 


"Code Inspection Simulation" 
Combines expert system and DVI to 
aid software code inspections 




Virginia Tech 


"NCR Product Theater" 
Displays NCR product information to 
d^onstrate DVI capabilities 




Videodisc Publishing 




"Design and Decorate" 
Facilitates interior design work 
using changeable fabrics, forms, 
room layout^ etc. 



Table 2. DVI Project Classifications 



DVI'Reiated Information. Information in this 
section of the interview constituted the major portion of 
the interview, and was structured around the five topics 
listed above. 

L WhatDVI'based application was being developed. 
Most of the DVI applications were being developed as 
prototypes — cither proof of concept or point of 
information {rojects; however, three of the applications 
were intended for commercial ventures. Table 2 shows 
classification of the projects as commercial or 
experimental and the titl^urposc of the project. In the 
year since our interview was conducted, some of these 
applications have come to fruition, while others have 
been abandoned. 

2. How it was done. 

The tasks involved in the development phase of the DVI 
{^plications varied from project to project The more 
ambitious projects, especially diose producing a 
commercial product, ten^ to follow a typical software 
development life cycle: planning, design, prototyping, 
implementatxm,* testing, and revision. In addition, most 
of the DVI sites also included two additional tasks in 
the development process: taking time to learn the 
hardware and software; and planning, i^eparing, and 
organizing source materials to put on CD*ROM. Some 
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Compounding the problem was the software support 
availability. Many developers had difficulty getting 
solutions to problems that were obsu^cting progress on 
their programs, because of understaffed supjxxt offices 
and an inefficient bulletin board for contacting support 
personnel. 

Other problems with DVI application development were 
software-related, but some hardware problems still 
existed at the time of our interview. Table 3 shows 
some common hardware and software problems in DVI 
application development encountered by our 
interviewees. Developers attributed the underlying 
causes of these problems mainly to the newness of the 
technology. They felt that most of the software and 
hardware problems will be corrected as the technology 
evolves, as will the problems with documentation and 
support. In addition, developers also attributed some of 
the problems to a lack of methodology for DVI 
development. Most were unaccustomed to the wide 
range of diverse backgrounds — software engineers, 
human-computer interface designers, video production 
specialists, instructional designers, graphic artists, and 
so on — needed for DVI development, and therefore felt 
that their methods for developing a future DVI 
application would change. Lastly, most developers felt 
that interactive development tools would greatly 
increase the efficiency of the DVI development process. 

4. Development tools. 

DVI application programming is an inhcxntly complex 
process and, without interactive tools, is very slow and 
cumbersome. Application developers need a set of 
multimedia development tools to facilitate production of 
DVI applications. In particular, the need for an 
interactive tool to alleviate some of the coding needed 
for basic DVI functions was indicated by many 
interviewees. The "DVI tools" that currently are 
supplied with the DVI support software are actually 
low-level programming support commands that require 



a detailed understanding of DVI. They do not facilitate 
application design and are usable only by a person — 
typically an application programmer—who understands 
the intticacies of DVI technology. Because of the large 
number of roles involved in developing a DVI 
application, the non-programming roles in particular 
need tool-based support to remove them from the chore 
of writing source code. 

Many of the interviewees had diverse definitions of what 
a comprehensive tool to support DVI development 
should be able to do, but most thought that a set of 
DVI development tools should exhibit several critical 
capabilities: 

• the ability to incorporate basic DVI functions 
into an application interactively — that is, 
without writing source (jHOgramming) code — 
and then augment functionality through a high- 
level programming language; 
a quick way to prototype the DVI i^lication's 
human-computer interface; and 
usability by non-programmers so that« for 
instance, graphic design c^n be don2 by a 
graphic designer. 

In addition to these capabilities in the development 
tools, interviewees also indicated that DVI programmers 
need o'Jier basic tools, such as a debugger to support 
programming efforts, an audio/video data editor, a 
special effects editor, and so on. These tools are 
necessary to improve the efficiency of DVI application 
developers and, ultimately, reduce development costs. 

5. General impressions. 

This section of the survey investigated each 
interviewee's impressions of DVI application 
development, as well as DVI technology in general. In 
sum, every person interviewed thought that their project 
was a success. 



Software Problems 



* Multiple AVSS files with audio streams open simultaneously causes system to crash 

* Edit Level Vidr;o (bLV) not working correctly; causes hard disk failure 
► Memory fragnr entation 

* Not enough memory available for large-sized application 

* Unpredictable i>VI function calls 

' Cannot read fron: hard disk while motion video is paused 

* Bugs more obscure for DVI application development than for "normal" applicatioPo 

* 256 X 240 graphic resolution when integrated into motion video 



Hardware Problems 



* Recommended hardware is the only supported hardware 

' Rapid succession of mouse events causes refresh of mouse pointer to lag behind 

* Synchronization of audio/video 

► Color distortion 

> Board conflicts 



Table 3. List of DVf Development Problems 
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All said that they accomplished what they had set out to 
do, whether it was proof of concept, point of 
infonnation, or a comniercial product When asked to 
list the good things about developing a DVI 
application, the responses included: 

flexibility of the DVI environment; 

• a digital environment that provides full- 
motion, full-screen, high quality video; and 

• the combination of video, audio, graphics and 
text into one format and medium. 

All interviewees saw DVI as having a potentially huge 
impact in training, commercial, and educational 
environments. However, interviewees felt that technical 
problems, especially the lack of documentation and 
suppoit, caused most of their delays. At the time of the 
interview, some interviewees suggested that new DVI 
projects wait at least six months for the technology to 
stabilize. 

Conclusions 

Through our survey results, it is evident that DVI has 
won approval from its developers as a viable and 
desirable technology. In particular, the following 
conclusions were drawn from our interviews: 

• Currently, DVI development is very slow and 
vciy difficult 

• DVI's greatest deficiency presently is lack of a 
development environment and trained 
developers. 

• Current DVI hardware and software still need 
much work» to improve their integration, 
functionality, usability, and reliability. 

• Interactive tools to support DVI development 
are virtually non-existent (notable exceptions 
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include MvJiaScript from Network Technology 
Corporation and Authology: Multimedia from 
CEFT), and will be difficult to build at a high 
level; however they are desperately needed 

• DVI development requires a broad variety of 
skills, some of which are not typically found 
in traditional software development 

• DVI project management must be done very 
adaptively. 

• DVI has an enthusiastic following; developers 
will build upon existing low level support, and 
are anxious to produce a range of products. 

In sum, results of our survey of ten DVI developers 
indicated that DVI has great potential as a multimedia 
technology, but it needs much more development 
support in order to be produced in an efficient, cost- 
effective numner. 
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MANAGEMENT ACTIVITIES FOR INTERACTIVE VIDEODISC PROJECTS 



Hueyching Janice Jih, University of Georgia 



Abstract 

Project management is an issue that has been neglected 
by the instructional system development (ISD) field, 
especially for newer instructional technology like 
computer-based interactive videodisc (CBIVD) projects. 
There is a need for new comers to translate and apply 
concepts and techniques into the execution of actual 
tasks in each sophisticated CBIVD project. Integrating 
the application of various skills needed in developing 
CBIVD programs demands management sophistication. 

This presentation will integrate discussion of 
management activities from personal experiences and 
from literature that has been available with the activities 
required to develop CBIVD projects. A project 
management activities matrix, suitable for CBIVD, 
forms the architecture of the presentation. The matrix 
outlines four management arrays through five project 
phases, analysis, design, production, implementation, 
evaluation, will be addressed: planning, controlling, 
communicating, and documenting arrays. 

Introduction 

Suppose you are an instructional designer, have already 
gained lots of knowledge about instructional theories, 
instructional system development (ISD) models, and 
instructional design principles, experienced in the design 
and development of instructional/ training materials, but 
not in the development of computer-based interactive 
videodisc (CBIVD) projects. You're in charge of a 
project to develop a CBIVD training program. The 
budget and time lines seem pretty tight, but you're not 
quite sure. You need help to manage this project, but 
where do you turn? How could you connect the 
discipline in instructional design (ID) with the real- 
world in order to execute tasks within a sophisticated 
IVD project? Or, suppose you have great computer- 
based management tools in hand, what events or 
activities should you put into each blank screen and 
when. To produce a CBIVD instuctional system is 
neither a small nor simple task. For most instructional 
design/development (ID) people, the search for guidance 
would be frustrating. 

Project management is an issue that has been neglected 
by the instructional d'*<Jien (ID) field, especially for 
newer instructional technology like CBIVD. The 
instructional system design and development (ISD) 
models available in literature are more conceptually- 
oriented than operationally-oriented. 71iese models do 
provide strong background on theories and highlight 
what must be done in the whole ISD processes. But, 
there is a need for new comers in computer-based 
interactive video project to translate and apply concepts 



and techniques into the execution of actual tasks in each 
CBIVD project. Furthermore, CBIVD projects require 
the services of a team with sophisticated skills: video 
production, programming, script writing, graphics 
design, instructional design, at a min inum. Integrating 
the application of these skills demands management 
sophistication. Schiffman (1986) mentioned that 
"whenever instrucl:>> 'j being designed, all five of the 
major categories arc called into play.** These categories 
are theories, system analysis, diffusion, consulting or 
interpersonal relations, and project management. In 
order words, management is an essential component in 
the ID discipline as well as in application. 
Unfortunately, while looking through major ISD 
textbooks we will find out that most of them don't put 
the management issue into account. Knirk and 
Gustafson (1986) said that there are five management 
activities—planning, organizificf, coordinating, 
evaluating, and rqx>rting in managing projects. They do 
give us a good picture about what management 
components should be. Nevertheless, they don't support 
it with detailed events for the readers usage. Because 
good management could not only facilitate successful 
accomplishment of project events within plans, but also 
arouse CBIVD team members* potentials and team 
producUvity, we do need an operational CBIVD 
management model which covers management activities 
in detail for each member and is familiar to team 
members as well as sponsors to serve as a 
communications framework. 

In 1988 and 1989, Greer published a series of ID project 
management articles for project manager's usage. Greer 
presented a ID Project Management Model which is 
divided into three phases and included ten steps: project 
planning, instructional development, and follow up 
phases. From the author's point of view, project 
management is every member's responsibilities rather 
than the manager's only. Hence, we need a management 
guideline which covers managing activities for each 
CBIVD team member and is familiar to members as 
well as sponsors to serve as a communications 
framework. An appropriate frame vork for managing 
projects should be common ID model instead of 
inventing any new framework which forces readers to 
reconstruct their mental models. This article integrates 
discussion of management activities from personal 
experiences and from literature that has been available 
with the activities required to develop CBIVD projects. 
A project management activities matrix, suitable ^or 
interactive videodisc projects, forms the architecture of 
the presentation. The mauix outlines four management 
arrays (cf.. Figure 1) through five project phases, 
analysis, design, production, implementation, 
evaluation, will be addressed: (a) Planning array, to 
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arrange the conditions and prepare plans for project 
activities; fl>) Controlling array: to stimulate, monitor, 
intervene, criticize, and modify project activities, 
materials, or products; (c) Communicating array: to 
deliver communications among team members and 
outside people; and (d) Documenting array: to document 
materials or products. Management activities in each 
array are described in following paragraphs. 



rigure 1. Four arrays of CBIVD project management 

Management Activities 

The project management task consists of planning, 
controlling, communication, and documentation arrays 
across five common ID phases (Figure 2). The four 
management arrays and their functions as well as 
activities are: 

Planning. The function of planning is to arrange 
conditions under which project activities take place and 
to prepare plans for how, when,a and by whom 
executing what events where. Planning activities in 
each phase are: 

Analysis Phase. The conclusion of previous problem 
analysis that instruction by CBIVD was the answer to 
the problem. The preliminary project plan was iq)proved 
by sponsors or higher-level authorities. 

To refine the preliminary project plan be estimating 
detailed schedule, cosu as well as task charts. This 
refining plan process and preparing preliminary plan 
could be done and revised conveniently by software 
tools. For instance, MacProject could be used lo plan: 
schedule chart, resources/task/project timetables, and 
costs. Managers, instructional designers (IDers), 
programmers, subject matter specialists (SMEs), and 
design/production artists should be the core group in ID 
projects. 

In order to accomplish the planning process, a manager 
needs to locate resources and to group the project team. 
For the purpose of communicating with others and 
record keeping, the core group should develop format 
sheets for controlling and communication. 

When finish each phase, the manager should deliver the 
revised materials or products to sponsors or higher- level 
authorities and gain written iq}proval from them in order 
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to set up operational conditions for the following 
activities. Finished project plan, ready-to-go project 
team, and sponsor approval are the outputs. 

Design Phase. For arranging the design environment, 
the core group cooperates to set up facilities, 
equipment, materials, and media. 

In order to control project activities with timetable, 
team members must prepare operational conditions for 
the next project phase. During the design phase,thc ID 
team needs to schedule the production process. At 
production phase, team members should prepare for 
implementation. 

Operational conditions for project design activities, 
contract with production house, and sponsor*s approval 
are the outputs. 

Production Phase. Set up facilities,, equipment, 
materials, and media for production and pilot testing. 
Arrange subjects and resources for pilot test 

Operational condition for production and evaluation 
activities, implementation plan, and sponsor's approval 
are outputs of this phase. 

Implementation Phase. Install hardware, software, and 
videodisc supporting with adjunct materials for 
implementation usage. Outputs are operational CBIVD 
courseware systems and sponsor's iq)proval. 

Evaluation Phase. Send formative and summative 
evaluation reports as well as impact evaluation plan to 
sponsor. Outputs are impact evaluation plan and 
sponsor's iq}proval. 

Controlling* The function of controlling is to 
stimulate, monitor, intervene project events (for 
manager); to criticize and modify project activities, 
materials, as well products; and to record activities 
during the development processes (for team members). 
Critiques or formative evaluation should start at the 
very beginning and go through the whole project. 
Modification or revision is the action which responds to 
evaluation results. The controlling funcuc is the same 
for each phase. The difference is activities or products to 
be controlled. The following list of activities and 
products to be controlled could form a basis of 
checklists. The detail about how to evaluate each 
activity or product is beyond the purpose of this article. 

Activities in Each Phase. 
Analysis: 

• Gather and analyze data for needs and learner 
analysis. 

• Locate resources, materials, equipment, 
facilities, media. 

• Develop content report. 



Planning 

To arrange the conditions 
and prepare plans for 
project activities 

Communicating 

To deliver 

communications among 
team members and 
outside people 



Controlling 

To stimulate, monitor, 
criticize, and modify 
project activities 

Documenting 

To document materials or 
products 
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PL 


ANNING 




Analysis 


Desien 


Production 


Implementation 


Evaluation 




Eslimalc 

time/budget 
Group projcclicam 
Develop project 

plan 
Design formal 

sheets 
Gain sponsor 

approval 


Schedule production 

process 
Prepare for 

production & 

testing 
Set up design 

facilities 
Gain sponsor 

approval 


Prepare 
implementation 
plan 

Set up production 
environment 

Set up pilot test 
sites 

Gain sponsor 
approval 


Install courseware, 

hardware 
Deliver adjunct 

materials 
Gain sponsor ^proval 


Gain sponsor 

approval 
Prepare impact 

evaluation 

report 


CO 


NTROLLING 




Analysis 


Desien 


Production 


Implementation 


Evaluation 




Stimulate activities 
Review/revise 

materials 
Takeiecords 


Stimulate activities 
Control activities 
Revise materials 
Revise subroutines 
Take records 


Stimulate activities 
Control activities 
Revise materials & 

products 
Revise courseware 
Talce records 


Stimulate activities 
Control activities 
Adjust CBIVD 

systems 
Takereoords 


Stimulate 

activities 
Control project 

activities 
Takcrcooids 


CO 


MMUNICATING 




Analysis 


Desien 


Production 


Implementation 


Evaluation 




Conduct meetings 
Cbntact outside 

resources 
Contact sponsor 
Informal 

communication 


Conduct meetings 
Deliver status 

memos 
Contact sponsor 
Informal 

communication 


Interact with 

production house 
Conduct meetings 
Deliver status 

memos 
Contact sponsor 
Informal 

communication 


Interact with 

implementation 

sites 
Conduct meetings 
Deliver status memos 
Contact sponsor 
Informal 

communication 


Interact with 

users 
Deliver rqxxts 

to sponsor 
Infonnal 

communication 
Conduct 

reflection 

meetine 


DO 


CUMENTING 






Analysis 


Desien 


Production 


Implementation 


Evaluation 




Needs assessment 

report 
Learner profile 
Cbntent report 
Budget/schedules 
Charts/forms 
Evaluation 

strategies 
Communication 

strategies 
Resources 

information 
Project plan 
Contracts 
Sponsor approval 
mijtds j 


TrcaUDcnt 
description 

Screen design 

description 

Format sheets 

Scripts 

Stor> boards 

Flowcharts 

Subroutine 
prognuiis 

Contracts 

Sponsor approval 
rcxxids 

Other materials 


Graphics 

Computer programs 
Video, audio tapes 
Master videodiscs 
Integrated 

courseware 
Adjunct materials 
Sponsor approval 

records 
Other materials 


Copies of software, 
videodiscs, adjunct 
materials 

Sponsor approval 

Records 

Other materials 


Finai reports with 

fccmative and 

summative 

evaluation 
Sponsor approval 

reoords 
Other materials 



Design instructional and testing strategies 
(micro design). 
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Figure 2. Management Activities for CBIVD projects 
Design: 

• Conduct treatment & fomauing design. 
O 312 
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• Write scripts and sloryboards. 

• Design gnphics, program subroutines. 

Production: 

Develop graphics and programs in computer. 

• Produce and edit video, audio, audio/video 
tapes. 

• Produce r master videotapes. 

• Produce check videodisc, master videodisc. 

• Integrate computer program with videodisc. 

• Develop adjunct materials. 

Implementation 

• Install Implementation system. 

Evaluation 

• Finish formative and summative evaluation 
reports. 

• Conduct impact evaluation activities. 

Products of Each Phase 
Analysis: 

• Needs assessment report 

• Letoner profile. 

• Budget, timetables, schedules, task charts. 

• Evaluation and communication strategies with 
format sheets. 

• Content reports (task analysis, outlines, 
objectives, etc.) 

• Project plan 0>roject description, schedule, 
charts, and reports). 

Design: 

• Treatment and formatting description. 

• Instructional algorithms. 

• Lesson design and testing strategies. 

• written scripts and storyboards with graphics 
design. 

• Program flowcharts and subroutines. 

Production: 

• Computer graphics, programs and codes. 

• Tapes and videodiscs, 

• Integrated courseware. 

• Adjunct materials. 

Implementation: 

• Copies of software, videodiscs, adjunct 
materials. 

• CBIVD courseware systems. 

Evaluation: 

- Final project report 

• Impact evaluation plan. 

Communicatittg. The function for communication 
is to communicate with each others among team 
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members and outside sponsors, experts, subjects, or 
other resources by informal and formal communication 
channels. The formal communicating ways for team 
members could be (a) by meetings— from the kick-off 
and brainstorming meeting to the reflection meeting; 
and (b) by project status memos through the processes. 
Formal communication between ID team and outside 
people are: interact with si)onsor, higher-level 
management, producers asiu applicators, and users. The 
informal communication may happen at any 
interpersonal interaction, such as, by telephone, 
conversation, rumcM*, personal observation, gestures, or 
feelings. Any interaction regarding project progressing 
should be recorded and analyzed 

Documentation. The function of this component is 
to document e^ ^ry materials, products, or programs (cf.. 
Figure 2) which are developed in planning, controlling, 
and communicating component 

Conclusions 

The appeal of CBIVD, with its myriad opportunities for 
instructional innovation and creativity, has encouraged 
ID professionals to push ahead This presentation offers 
some suggestions for IDers to clarify and explain the 
project management process and tries to identify 
essential management components ir. CBIVD projects. 
As we know, no list can be all-inclusive, because every 
projects is imique while compared with others. Many 
other activities may be added as projects become more 
complex and several activities could be moved out when 
tasks become simpler. However, it the the author's hope 
that the model provides applicable starting points for 
readers and handouts delivoed in the presentation will be 
provided in support of the matrix to guide the novice 
CBIVD project manager during their first experiences 
with CBIVD development 
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USING AN EXPERT SYSTEM SHELLS AS AN AUTHORING TOOL FOR INTERACTIVE 

VIDEODISK APPLICATIONS 



V. Pigford, Western Kentucky University 



Abstract 

This paper presents the usage of an expert system sh**,!! 
as an authoring tool for the implementation of 
interactive videodisc applications. The power of the 
inference engine combined with informational databases 
and external program calls make selected expert systems 
shells excellent authoring tools for videodisc 
prototyping and implementation. One microcomputer 
based expert system shell. VP-Expert, is discussed in 
detail 



could utilize an expert system to diagnose and remediate 
first year algebra problems. An expert system can be 
integrated with the curriculum to assist in tutoring in a 
particular area. The potential for using expert systems 
as an intelligent tutor is a catalyst in educational 
technology today. 

These examples indicate that the purpose of an expert 
system is iO assist in a particular domain of human 
activity. General classifications of expert systems 
(Frenzcl, 1987) arc given in Table 1. 



Introduction 

The purpose of this paper is to explain and document 
the feasibility of using an expert system shell as the 
vital authoring tool in microcomputer based videodisc 
applications. The papci is divided into the following 
parts: 



1. 
2. 



3. 
4. 
5. 



OVERVIEW OF EXPERT SYSTEMS 

INTEGRATION OF EXPERT SYSTEMS 

AND LEVEL ni VIDEODISK 

IMPLEMENTATIONS 

VP-EXPERT AS AN AUTHORING TOOL 

SUMMARY 

APPENDIX A: OVERVIEW OF VP- 
EXPERT 



In section one, the interested reader will find a quick 
overview and definition of ternis used in expert systems 
and intelligent tutoring systems. Section two will focus 
on the features of an expert system shell that enhance 
the implementation of interactive video. A technical 
overview of how the concepts in part two can be 
implemented with VP-Expert, a microcomputer based 
expert system shell, is discussed in section three. The 
next part will summarize the major points of the entire 
paper, and the appendix will give an overview of VP- 
Expert 

Overview of Experts Systems 

An expert system is a software program that mimics the 
reasoning ability of some human expert in solving a 
particular set of problems. The problem or domain area 
is nanow in scope as opposed to general problem 
solving. An expert system could give advice on 
selecting a money market plan, assist an aircraft 
mechanic on repairing a Boeing 707, diagnose the 
medication for bone cancer, or generate possible options 
for vacations in Hawaii. 

Expert systems can enhance educational situations. A 
principal could use an expert system to assist in 
designing student schedules. A mathematics teacher 



In the area of computer-based instruction, expert 
systems can control a large instructional database, d^ug 
student PASCAL programs, design educational 
products, diagnose mathematical difficulties, teach 
computer literacy, interpret current events, plan for 
minority education, predict kindergarten eniollinent, or 
repair microcomputers. The main area of interest for 
this discussion is computer controlled or level III 
interactive video and audio instructional systems. 

Table I. Classification of Expert Systems 



Control 


intelliitent automation 


DebuKKinK 


recommends corrections to faults 


Design 


development of products by 
specifications 


Diaj^iosis 


Estimation of defects 


Instruction 


Optimized instruction 


Plannini^ 


Development of Roal-oriented schemes 


Prediction 


Intelligent Ruessins of outcomes 


^pair 


Automatic troublesnootinj! and fixing; 



Any expert system logically has three parts (Pigford & 
Baur, 1990): a rule base, an inference engine, and a user 
interface. These three logical components are given in 
Figure 1 below. 

The rule base is a set of rules that represent the 
knowledge of the human expert. Generally the rules are 
expressed in an IF THEN format. The inference engine 
is t^e reasoning mechanism that analyzes the rules to 
reach a conclusion. The user interface is the total 
communications between the expert system and the user 
of the intended expert system. Together, correct rules, 
an automatic reasoning strategy, anc': effective 
communications make an expert system efficacious in 
the intended domain. 

Since the expert system is a piece of computer software, 
how is it written? The answer is basically in two ways. 
The expert system can be programmed by a programmer 
using any language: PASCAL, C, Fortran, or most any 
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high level language. The second method of 
implementing an expert systeni is to use a tool called 
an expert system shell. The shell is easier to use and 
does not requif' the author to be a professional 
programmer. 



system shell) can control the branching in an 
instructional application. Generic rules could be as 
follows: 

Rule 29 



u 
s 

E 
R 



Knowledge Base 



Inference Engine 



INTERFACE 



Figure 1. Three pans of an Expert System 

The shell contains the automatic reasoning ability or 
inference engine. Hence, the user of the shell has to 
supply only the rules and the interface. This makes it 
possible for non-programmers to implement expert 
systems in a selected area or domain of expertise. There 
are many expert systems shells available that run on 
miaocomputers, minicomputers, and mainframes. For 
interactive video, however, microcomputer based shells 
are the most useful, affordable, and prevalent 

In summary, expert systems are software programs that 
display the reasoning ability of a human expert in 
dealing with a specific problem. Experts syster.is arc 
composed of three logical parts; the rule (knowledge) 
base, the inference engine (reasoning mechanism), and 
the user interface (communications). The expert system 
software can be written in a computer language by a 
programmer or implemented with a shell by a non- 
programmer. 

Integration of Expert Systems and Level 
III Videodisc Applications 

Expert systems can be integrated with Level III 
videodisc applications. As proposed by Baur (Baur, 
1988), expert systems can actually be the central or 
driving force for interactive video and audio peripherals. 
Prototype systems (Devoldcr, 1989; Ting, 1988) have 
been built using an expert f;ystem shell to implement 
intelligent tutoring via a microcomputer based videodisc 
system. 

The rule base of the intelligent tutor (expert system) 
contains the intelligence for diagnosing, interpreting, or 
planning the learning outcome. The automatic 
reasoning mechanism (inference engine of the expert 



IF Achievoiiient = Low THEN 
Play Videodisc Section R 
Give Posttest 
Record in the Database 

In this rule, if achievement is low, then the inference 
engine will instruct the system to play a particular 
section of the videodisc, administer a posttest, and then 
record the results in the student's record in the database. 

Another application might make use of the following 
rule: 

Rule 234 

IF Leaming^style = Graphic and Motivation = 

High THEN 
Play Videodisc Section E with Graphics 

This nde tells the inference engine to play a particular 
section of the videodisc with overlay graphics turned on 
when the student is highly motivated and possesses a 
graphical learning style. If an expert system had several 
hundred rules, then the inference engine of the expert 
system shell would examine all rules to determine the 
appropriate action of the tutoring system. 

In the two examples above, the expert system had to 
access the videodisc, turn environmental controls on or 
off, administer tests, and record results in a database. If 
would be much more efficient and economical in the 
long run, if a new test or new database did not have to 
be written within the expert system. It would be 
beneficial if the expert system could behave like a 
maximal authoring system to communicate with other 
pre-existing software (Pigford, Henry, & Miller, 1984). 
Another way to express this concept is to require that 
the intended authoring tool have an "exit and return" 
feature. Some expert system shells possess this 
desirable feature. 

This integration of expert systems and videodisc can be 
perceived as a wheel with the expert system as the 
center of the wheel and the various software packages as 
the spokes. Sec Figure 2., 

The core of the integrated system would be an expert 
system that controlled the learning environment 
(videodisc, graphics, audio, etc.), and utilized 
informational systems (database, spreadsheet, statistical, 
etc.). As the application varied, the utilization of other 
software would vary. 

The integrated system would be much easier to produce 
if the expert system shell allowed this integration of 
external software. An expert system shell with its 
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automatic reasoning (branching) mechanism that could 
utilize external software programs would be an efTectivc 
authoring tool for Level III videodisc applications. In 
the next section VP-Expert will be discussed as an 
authoring tooL 




Figure 2. The Integration of Expert System and 
Videodisc. 



VP*Expdrt as an Authoring Tool 

The focus of this paper will now shift from the general 
concept of using an expert system as an authoring tool 
to the specific implementation of one microcomputer 
based shell called VP-Expert VP-Expert is an expert 
system shell than runs on IBM compatible 
microcomputers under DOS 2.0 or higher. The inference 
engine will be discussed first The FIND command is 
the instruction in a VP-Expeit program what turns 
control over to the inference engine to search the rules 
or knowledge base to reach a conclusion. If the value of 
a variable GRADE is unknown, then the instruction 
FIND GRADE will turn control over to the automatic 
reasoning mechanism to search the rules to determine a 
value for GRADE. If the inference engine finds a rule 
with GRADE as the conclusion, then the inference 
engine will look at the rule condition. For example in 
the following rule, 



expert system shell. With the appropriate rules from the 
designer of the videodisc application, the inference 
engine serves as the automatic **branching** system for 
an authoring tool. 

The second major feature for an authoring tool is the 
ability to integrate or utilize an informational base and 
other pre-existing and specialized software. VP-Expert 
has these features built in the shell. VP-Expert can 
utilize information previously stored in database, 
^readsheetand text files. Hiis external data can be read 
into VP-Expert and actually incorpcnited in the rule 
base. The MENU instruction along with GET is used to 
access a daubase record from VP-Info, dBASE II, 
dBASE III, dBASE m-i-, or other work alike daubase 
management programs. The PUT instruction will write 
infomiation from VP-Expert back to the database. The 
APPEND instruction will allow VP-Expert to insert a 
new record into the external database file. Other 
commands will effect the same integration for 
spreadsheet access to VP-Planner, VP-Pianner-Plus, and 
LOTUS 1-2-3. Text files can also be transferred to and 
from VP-Expert This integration with external 
infcmnation bases is a desirable feature of an efiective 
authoring tool. 

The flexibility of VP-Expert is further exemplified by 
its ability to call external programs and then return to 
VP-Expert This **exit and return** feature increase the 
flexibility of VP-Expert in iq)plications development 
with videodisc. The user can actually switch from the 
expert systems interface to spreadsheets with WORKON 
and then return to the expert system tutor. 

VP-Expert can communicate with any external DOS file 
that has a .BAT, .COM, or EXE extension. A EAT or 
batch file is a file of DOS (operating system) 
comnumds that are interpreted one after another as if 
they were typed at the A> prompt bom the keyboanL A 
DOS file with a .COM extension is an executable file 
that is limited to about 64k whereas a DOS file with an 
EXE extension is an executable file that is not limited 
in size. (For example. Turbo PASCAL, ver 5.5, 
produces EXE fiies). In any manner, any of these DOS 
files can be integrated by using the BCALL, CCALL, 
and CALL commands of VP-Expert For example, the 
generic rule discussed in Section two converts to 
^ific VP-Expert code as follows: 
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RULE 45 

IF Score > 90 and Assignments = Complete 
THEN GRADE = A; 

the inference engine would now examine the values of 
Score and Assignments. If the values are found, then the 
inference engine could determine if GRADE is an A. If 
the values of Score and Assignments arc not found, then 
the inference engine would examine any rules that 
would determine Score and Assignments. In any case, 
the path of the inference engine is automatic in an 



RULE 29 

IF Achievement = Low THEN 
CALL Video,*'start, stop" 
CALL Test 
CALL DB,"Rcsults"; 

If rule 29 is activated then the expert system calls an 
external file, VIDEO.EXE extension which plays the 
videodisc from the start frame to the endi..^ frame 
number. Then the inference engine calls on the test 
program, TEST.EXE, and finally the results of the test 
are transferred to the DB program. The code above 
assumes the existence of the three executable files 
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named in ihc ihrec CALL instniciions. ^VP-Expcri docs 
not have its own videodisc library.) The important 
consideration is that these three executable flies could 
have been created with many different software 
compilers or packages. 

The second «ale from the previous section would 
translate into the following VP-Expert instructions: 

RULE 234 

IF Leaming_style = Graphic and 
Motivation = High THEN 
BCALLVidcog 

This rule, when activated by a true condition, would 
execute a batch flle which would send instructions to 
the videodisc to play a certain sequence with graphics 
overlay on rather than off. Moreover, VP-Expert will 
assist in controlling the screen during these external 
program calls (to the videodisc, cpreadsheets, database, 
or specialized software). The instructions are CLROFF 
and CLRON. In the following sequence of code, 

DISPLAY **Please wait a minute for the 

calculation.** 
CLROFF 

CALLSTAT,"data" 

DISPLAY **Statistics are being calculated.** 
CLRON 

CALLGr^*'data*' 

the user is being told to wait a minute while the 
statistic from the tutoring session is figured. The 
CLROFF command and the subsequent CALL to 
STAT£XE allow the program to run tranq[)arently to 
the user. When Jte statistic is calculated, the CLRON 
command reactivates the screen so that when 
GR^VPH^XE is called the screen will display the 
statistics with the GRAPH program. The VP-Expert 
commands to control the screen make the intermixing 
of programs almost transparent to the user of the 
intelligent tutoring system. Hence, through the use of 
the automatic inference engine, access to information 
bases, and external programs calls, VP-Expert can 
function as an authoring tool for interactive videodisc 
applications on IBM compatible microcomputers(The 
interested reader is referred to the Appendix which 
contains additional features of VP-Expert such as 
Graphics, Hypeitext, and Chaining.) 

Summary 

This paper has presented the relationship between expert 
system shells and authoring tools for Level III videodisc 
implementations. An expert system which emulates the 
intelligence of a human can be implemented with 
computer programming languages or a special tool 
called an expert system shell. Any expert system is 
composed of three parts: the rules (knowledge), the 
inference engine (reasoning or search mechanism), and 
the user interface (total communications medium). The 



Expert Systems in IVD 

expert system shell has an automatic inference engine 
which does not have to be programmed. In using an 
expert system shell to implement a Level III videodisc 
system, the automatic inference engine controls the 
^ranching in the instructional system. The ability to 
access external information bases and to utilize external 
software as an **exit and return** feature can uanspose an 
expert system shell to an authoring tool. VP-Expert has 
the specific commands to accomplish the same 
objective as an authoring tool. 

Appendix 

VP-Expert offers a combination of powerful features 
that make it ^^xq^riate for developing microcomputer- 
based expert systems. Special features include: 

• The ability to exchange data with VP-Info or 
dBASE database files, VP-Planner, VP-Planner 
Plus, 1-2-3, or Symphony worksheet files, and 
ASCII text files. 

• An Induce command that automatically creates 
a knowledge base from a table contained in a 
text, database, or worksheet file. 

• An *'infcrence engine** that uses backward and 
forward chaining for problem solving. 

• Optional development windows that let vqu 
observe the behind-the-scenes path of the 
inference engine as it navigates the knowledge 
base to solve problems during a consultation. 

• Confidence factors that let you account for 
uncertain information in a knowledge base. 

• Simple English-like rule construction 

• C^nuuids that allow VP-Expert to explain its 
actions during a consultation. 

Knowledge base **chaining,** which lets you 
create knowledge bases that would otherwise be 
too large to fit in memory. 

• A built-in text Editor. 

• Automatic question generation for finding 
unknown values of variables. 

• The ability to record and graphically display 
the rule-by-rule search pattern used behind the 
scenes during a consultation. 

• Floating point mathematical functions. 

• Rapid execution of the knowledge base. 

• The ability to execute external DOS programs. 
Hypc*xxt 

• Dynamic images. 

• SmartFonns. 

• Graphics primitives. 

• Mouse support 

In addition a Structured Qaery Language (SQL) interface 
and special gnqAics libraries can be added to VP-Expert 
VP-Er.pert is available in a ful version or an 
educational version. For more detailed informaticm 
contact 

Paperback Software International 
2830 Ninth Street 
Berkeley, CA 94710 
Phone (415) 644-21 16 
FAX: (415) 644-8241 
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Ms. Lori Peterson at Paperback Software International 
can give you more infomiation on educational accounts, 
whereas Mr. Terry Schussler can provide detailed 
technical infomuiuon. 
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IDENTIFYING RESISTANCE TO COMPUTER BASED TRAINING 



John C. LeDoux, FBI Academy 



Abstract 

This practical discussion will suggest likely reasons for 
resisting CBT, with particular attention being paid to 
trainers. The discussion will end with a strategy for 
overcoming resistance. 

Introduction 

Whenever you suggest a plan that may cause change in 
your organization, you need to become aware of who 
may resist the change and why they would do so. The 
implementation of computer based training (CBT) is, in 
fact, the introduction of major change. CBT requires the 
use of new technology, cooperation between 
departments that may not be used to working together, 
and redefining the role of some of the trainers. 

Over the past seven years Uie author has had the 
opportunity to speak with many CBT professionals. A 
high degree of agreement seems to exist with regard *o 
the identities of those likely to r?^st CBT and why they 
might do so. Surprisingly, the major opposition often 
arises from other trainers. This article will discuss 
potential resistors, strategies for overcoming this 
resistaiKe, and specifically examine how your fellow 
trainers may view CBT. 



Identifying Potential ResUitors 

Since successful change requires the involvement of 
employees, an initial step in'ov^a)ming resistance to 
any change such as the implementation of CBT is to 
identify whose resistance might be anticipated. The 
identities will change slighUy, of course, from 
organization to organization, but the basic premise for 
identifying the important player remains the same. 
Identify each person or group that has an interest in the 
cunent product, or would be iiQpacted by tfie proposed 
change. For CBT, students, /trainers, executive 
managers, and Une management should have an interest 
in the current training. The implementation of CBT 
would impact, to some extent onfall these g;oups and 
probably would impact also on the data processing 
dq^artment 

A complete analysis of possible resistance to change 
requires examining each person who has influence 
within each group. For npw, let us limit the analysis to 
identifying what might be some of the reasons each of 
these groups might rerist CBT.' 

Table 1 lists nine possible general causes for resisting 
CBT and the likely areas of high resistance for each of 
five groups. 



CAUSE OF 
RESISTANCE 


GROUP 




Top 
Mang. 


1st Line 
Mang. 


Trainers 


Students 


DP 
Group 


Cost/Benefit 


• 








• 


Loss of Job 




• 








Loss of Control 










• 


Dealing with Technology 








• 




Presumptions 


• 










Ignorance 


• 


• 








Isolation of Students 








• 




Implementation Problems 


• 








• 


Culture 


• 
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The executive managers, labeled Top Mang. on Table 1, 
are the ones in an organization from whom permission 
must be obtained before expending funds to purchase 
computers, consultant services,commerciaI CBT 
software, or CBT development software. Their concerns 
center around cost and the impact of the change of the 
organization. 

Thus, they are highly concerned about the cost-benefit 
ratio, the problems associated with the implementation 
of CBT, and the impact of CBT on the organizational 
culture. Also, they will be affected by their 
presumptions regarding training since CBT is different 
than the traditional model wherein an instructor leads a 
group of passive learners. Since CBT is a relatively new 
phenomenon within corporate training, there may be 
resistance due to their lack of familiarity with CBT. 

The line managers, labeled 1st Line Mang., do not have 
many reasons for resisting CBT.. Unfortunately, this is 
probably because many line managers are not that 
involved in training. As long as employees get the 
training they need and are away from their job for as 
short a time as possible, httle consideration is given to 
training. The major reason this group might balk at 
CBT is unfamiliarity with CBT. 

Employees who arc the students will be a little more 
feaiful of CBT. Mcne specifically it's the computer part 
of CBT about which they are concerned. It may be that 
there is some latent fear of computers taking over jobs, 
or it may be simply that they have never used a 
computer before and are afraid they may look foolish. 
Another significant concern is that they perceive CBT 
as working alone at a computer. One of the motivators 
identified for attending in-service training is the 
socialization that occurs in the class. The students do 
not want to lose this and are fearful of any change that 
may alter the existing training climate. 

The data processing (DP) personnel will be even more 
concerned* They are accustomed to being seen as the 
computer experts and are rather anxious to defend their 
territory*. Thus, they fear a loss of control in the 
computer area due to the implementation of CBT. They 
will be concerned about who will have the final word on 
hardware and other technical decisions. These fears and 
concerns are exacerbated by their ignorance in the area of 
CBT. Since the CBT project may possibly compete for 
funds with various DP projects there will be a natural 
concem on the part of the DP personnel with regard to 
the cost-benefit aspects of CBT. To a lesser extent, the 
DP personnel will be anxious about the direct threat to 
their jobs when outsiders start to become more familiar 
with computers, and the impact of CBT on the 
organizational cultiire. A final reason for resisting CBT 
may be Jie presumptions the personnel hold concerning 
training and the role of trainers. 

The group mo5:t threat£ ned by the change is the training 
staff. At first, this may seem surprising because trainers 
like to think of themselves as creative and innovative 



persons. There are, however, some ver> legitimate 
reasons why trainers would fear and thus resist CBT. 
First, their jobs are being directly affected by the 
proposed change. They will have to learn new skills and 
assume new roles. They fear they will lose some of the 
control they now have in "their" classroom. They may 
be uncomfortable with working with an emerging and 
constantly changing technology. Implementation of 
CBT may challenge their training philosophies, change 
the organizational culture, and isolate them from their 
students. Likewise, resistance may be precipitated by 
the potential increased workload implementation will 
cause and the normal resistance to change some of us 
possess. 

Identify Specific Reasons to Resist or 
Accept CBT 

Once you have identified the appropriate parties and 
identified their general reasons for resisting CBT, you 
need to more specifically identify what would motivate 
each person to either resist or support the proposed CBT 
effort. Different persons within the same group may 
have slightly different reasons for their attitude toward 
CBT. Therefore, prepare an analysis for eich person. 
Table 2 shows one form such an analysis could take. 



SUPPORT 



RESIST 



BENEFITS 



DRAWBACKS 



Table 2* Analysis of Potential Consequences. 

The key to understanding this chart is to understand that 
there are individual advantages and disadvantages to 
either supporting or resisting the CBT project. Each 
person will have reasons listed in each of the four 
quadrants, though the specific reasons will va.y from 
person to person. 

A person may decide to resist CBT because of the 
advantages they see in so doing (Quadrant 2), or due to 
the perceived drawbacks of implementing CBT 
(Quadrant 4). Similarly, a person may decide to actively 
support CBT because of possible benefits (Quadrant 1), 
or because of disadvantages of resisting the effort 
(Quadrant 3). Since each individual will likely have 
reasons in each of the four quadrants, some persons will 
choose to remain neuu^I on the subject. 

Three major groups, thus, will emerge. One group will 
actively support CBT and a second will actively resist 
CBT The third group consists of those who perceive a 
negative impact if they either actively support or 
actively resist CBT. 

The advantages for resisting CBT would be determined 
and identified in the upper right quadrant. These are 
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desirable results that should occur if a person is 
successful in resisting CBT. For example, a trainer 
would be able to continue to do what he/she liked best - 
stand-up Ifaining if CBT is not allowed to inuiide. 

The disadvantages of supporting the CBT effort would 
be listed in the lower left quadrant. These are results that 
will have a negative impact on the person who 
successfully supports the implementation of CBT. In 
many cases these are the reverse of the reasons listed in 
the upper right quadrant. For example, a trainer might 
see a negative aspect to using CBT since it will reduce 
the amount of time spent in giving lectures. 

The advantages for supporting CBT are listed in the 
upper left quadrant. These are the anticipated positive 
results if CBT is adopted. For example, a trainer would 
develop a new job skill due to working with the project 

The disadvantages of not supporting CBT would be 
listed in the lower right quadrant. These are anticipated 
results that may have a negative impact on the person 
who acti vdy resists CBT. Let's take a closer look at the 
trainers so that we may fully understand the potential 
areas of resistance and learn to overcome them. 

Trainers' Resistance to CBT 

At a class given by Effective Training Inc. CBT 
devetopers of varying levels of experience were asked to 
independently rate the potential resistance to CBT of the 
groups previously discussed. The collected results 
formed pan of the input for Table 1. As the developers 
shared their perceptions, it became quickly evident that 
the trainers, as a group, were the persons most likely to 
resist the implementation of CBT. The results of this 

SUPPORT 



BENEFITS 



DRAWBACKS 



Table 3. Analysis of Potential Consequences for Trainers. 



unscientific study are supported by the presentations 
given at various CBT professional meetings. A 
common topic is how to interest insuiictors in CBT. 

Trainers tend to be outgoing persons who are interested 
in helping students with whom they come into contact. 
They want their students to learn, and in theory, they 
are paid to do this. Why then do we find that trainers are 
often among those who most resist CBT? Because, as 
was noted above, trainers are the group most affected by 
the changes precipitated by CBT. 

Table 3 lists some reasons trainers may support or 
resist CBT. The chart is generic in that it attempts to 
capture numerous motivations. The chart you would 
prepare would be based on your perceptions of a specific 
individual. 

The chart is similar in format to Table 2, but includes 
an additional aspect - "Dimensions." Consider each 
reason for accepting or resisting CBT in terms of its 
potential impact on the person you are analyzing. There 
are three sets of dichotomies to consider. A reason is 
either 1) Personal or Organizational, 2) Immediate or 
Delayed, and 3) Certain or Speculative. 

A Personal reason is one that a person believes will 
have an impact on him/her as an individual. An 
Organizational reason will impact on the organization, 
but not especially on the person. The second dichotomy 
examines the question of whether the reason will likely 
cause an Immediate or Delayed result. The final 
dimension attempts to determine the likelihood of the 
results. 



RESIST 



New Job Skill (PIC) 
Seen as Expert (PDS) 
Seen as Innovator ( PDS) 
Do Job Better (PDS) 
Increase Influence (PDS) 
Possible Advancement (PDS) 


Keep Influence (PIC) 
Do What Good At (PIC) 
Keep Kudos (PIC) 
Less Work (PIC) 
Low Risk (PDS) 
Save Money (ODG) 


More Work (PIC) 
Difficult to Manage (PIC) 
Tedious (PIC) 

Reduced Student Contact (PDS) 
Delayed Praise (PDC) 
High Risk (PDS) 
WUl Fail (PDS 


Seen as Block (PIC) 
Embarrassed if Succeeds (PIS) 
Become Stagnant (PDS) 
Lost Opponunity (PDS) 
Fall Behind Others (PDS) 
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Benefits for Supporting CBT 

Quadrants 1 and 3 of Table 3 suggest several reasons for 
deciding to actively support CBT. Quadrant 1 lists the 
possibility that you will grow as a trainer by learning a 
new job skill. You may be seen as an innovator who 
possesses special expertise. This may help in terms of 
organizational influence and advancement and raises the 
possibility of building an "empire" based on CBT. 

And finally, a trainer who is truly concerned with 
improving the instruction provided to the students 
should be willing to explore new methods of delivery.. 
CBT will allow you to provide more specific 
instruction to a larger number of persons than you now 
serve. There are many potential benefits to the students 
and the organization of using CBT, and each of these 
benefits is an additional reason for supporting CBT. 

Quadrant 3 submits several reasons why there may be a 
negative impact to resisting CBT. These include falling 
behind others, lost opportunity, and continued 
stagnation. 

Benefits of Resisting 

Quadrants 2 and 4 list several reasons why trainers 
would want to actively resist the development or 
purchase of CBT. Quadrant 2 notes that traditional 
methods of instruction will allow the trainers to keep 
their personal influence. In a lecture setting the 
instructor is the focal point of the class. The trainer, 
thus, sets the pace and direction of the class. Who 
among us would deny that there is a tremendous ego 
satisfaction in standing in front of a class and receiving 
their approval? For example, one developer told of 
showing some commercial CBT to a trainer who 
thou^t it was quite good, but asked "Instead of buying 
a copy for each student, couldn't we just have one copy 
for me in the front of the room and I'll ask the students 
what answer they would like to enter?" The most 
important thing to this trainer was maintaining his 
influence in the classroom. 

A second reason for resisting is that most trainers have 
spent long years honing their skills as stand-up trainers. 
They are good at it, and naturally want to continue to do 
what they do well. We have achieved a certain status 
based on our classroom their skills and don't want to 
abandon those skills that have served them so well. 

Third, experienced trainers get to coast a little because 
they are experienced. They have polished their lesson 
plans and presentations to almost perfection. Remember 
the early years and all the time it took to get ready to 
give a class? Now, a few minor changes and a quick 
glance at the lesson plan suffices. Why waste all this 
effort and start from almost the beginning? CBT 
requires a lot of effort to produce and it's so detailed. 
Instructors can always be a little vague in class and if 
they have to they can fake it, but CBT can't be faked. 
It's just too much work to help produce CBT, 



particularly since trainers arc convinced that they arc 
overworked now. 

Fourth, the implementation of CBT involves both 
expending funds and taking risk. Two benefits of 
actively resisting CBT, therefore, occur. First, from a 
short-term perspective, you save money. And, you 
avoid taking any personal or organizational risk. 

And finally, a good reason to resist CBT is because it 
may take the students from us. One of the joys of 
training is the interaction trainers have with the 
students. Many people became trainers because they 
liked people. Being in a room where there are a lot of 
people sitting at computers woiicing independently does 
not seem that rewarding. Trainers may believe they will 
miss the warmth they used to enjoy and the appreciation 
of their students. 

Quadrant 4 also suggests some reasons for avoiding 
CBT. These include the delay of praise from students, 
the tedious nature of CBT development, and the 
potential difficulty in managing CBT. 

Drawbacks to Actively Taking a Position 

There are negative aspects of either actively resisting or 
supporting CBT. Resistance may cause one to be 
labeled as troublesome by management, or to fall 
behind more progressive colleagues. Support for the 
effort creates risk, and may cause some unpleasant 
results such as an increased work load or a reduction in 
interaction with students. 

Some persons, thus, may take a neutral position for one 
of two reasons. For some the perceived benefits and 
drawbacks to CBT may appear equal. They therefore do 
not take a position. Others iHay tend toward supporting 
or resisting the project, but fear the consequences of so 
doing. Such persons become passive supporters or 
resistors. 

Overcoming Resistance 

The information prepared above forms the basis of a 
strategy for overcoming resistance. The reasons listed in 
each of the four quadrants can guide you in d'^aling with 
each person. In order to gain active support identify all 
the reasons where the implementation of CBT should 
result in a perceived benefit. You should readily agree 
with the individual that the benefits they have 
recognized are significant. You may also share with 
them some of the other possible benefits that may 
accnie. 

Similarly, identify all the reasons why actively resisting 
CBT might be detrimental to someone. You would 
concur that there certainly might be some undesirable 
side effects of resisting CBT and attempt to heighten 
their awareness of these effects. You may well be able 
to interject some additional drawbacks of resisting CBT. 



ERLC 



323 

333 



LcDoux 



On the other hand, you would want to lessen the 
significance of the negative impacts they view as 
occurring if CBT is implemented. And of course, you 
would attempt to ally the fears they have that form the 
basis of reasons why to resist CBT. For example, you 
might point out that using CBT allows the instructor 
more time for individual interaction with students. 

Remember that the most influential reasons will be 
those that arc rated Personal, Immediate, and Certain in 
the dimensions column. You would, thus, attempt to 
especially bolster and support the person in these areas. 
And you would pay particular attention to overcoming 
reasons to resist that have these three ratings in the 
dimension column. The use of this strategy should 
enable you to increase the assistance of the supporters, 
obviate the objections of the resistors, and move the 
uncommitted to your point of view. 

Conclusion 

While change potentially is threatening to all those 
affected, some individuals will be more threatened than 
others. To help overcome resistance to CBT, you 
should first identify those who are likely to be potential 
resistors. Next you should attempt to evaluate specific 
reasons for supporting or fighting the development of 
CBT. And finally, use this knowledge to show the 
potential resistor the benefits that they can derive from 
supporting CBT. 
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Abstract 

Although musical ability tests have enjoyed a 70-year 
history of development and refinement, they continue to 
be plagued with problems of inefficiency, poor 
reliability and poor validity. The shortcomings of these 
tests are so serious that many music educators have 
become completely disenchanted with them and have 
sought alternative methods of evaluation. One 
promising alternative to conventional music testing is 
computerized adaptive testing. Research to date has 
indicated that adaptive music tests require considerably 
fewer items to match or surpass the reliabilities and 
validities of conventional music tests. Much of the 
superiority of adaptive music tests can be attributed to 
the large pool of items that an adaptive test can draw 
from. The puipose of this study was to assess the 
quality of an adaptive test of musical ability that used 
an item pool two to three times smaller than those used 
in previous investigations. Results indicated that the 
adaptive music test used up to 62% fewer items to reach 
reliabilities and validities comparable or superior to 
those of three conventional music tests. These findings 
suggest that adaptive tests can provide significant 
improvements in the assessment of musical ability even 
when limited-size item pools (90-100 items) arc used. 

Introduction 

Formal tests of musical ability tests can be traced back 
to the pioneering work of Carl Seashore at the 
University of Iowa with his Measures of Musical 
Talents (Seashore, 1919). Seashore assessed aural acuity 
in a variety of modes (pitch, duration, intensity, timbre, 
rhythm and tonal memory), emphasizing the 
relationship of perceptual ability in musical 
accomplishment. His work sparked a sizeable amount of 
research and inspired the development of several 
additional musical ability tests (Bentley, 1966; Drake, 
1957; Gaston, 1957; Gordon, 1965; Wing. 1961). The 
most notable of these is the Musical Aptitude Profile 
(1965) by Edwin E. Gordon, which acknowledged by 
several music testing experts (e.g., Buros, 1972) as 
the best musical ability test currently available. Since 
1965. Gordon has developed additional tests for young 
children (1978), but the basic approach to music 
aptitude testing has not changed, despite advancements 
in technology and testing theory, since the time of 
Seashore. 

Throughout their long history, musical ability tests 
have been used primarily in music research studies but 
have not enjoyed widespread popularity among 
practicing music educators. This is ironic because music 
educators should benefit most from the infomiation 



provided by tests since test results could be used to 
diagnose music learning difficulties, detect discrepancies 
between achievement and ability, and identify 
individuals who arc likely to become excellent 
musicians. Although one reason music educators do not 
use musical ability tests is that they are unaware of 
them, the primary reason is that the tests themselves 
have shortcomings. Even the best commercially- 
available musical ability tests are inefficient and often 
show evidence of only low to moderate reliabihty and 
validity (Shuter-Dyson & Gabriel, 1982). These 
shortcomings have led some music educators to abandon 
these tests and to seek other methods of evaluation 
(Davies, 1978). Until the advent of computerized 
adaptive testing, however, no superior alternatives 
emerged. 

Adaptive testing procedures may help overcome some of 
the problems of inefficiency, poor reliability, and poor 
validity that have plagued music tests for the last 70 
years. Prior research on adaptive music tests with large 
item pools (180 or more items) has shown that adaptive 
tests using up to 69% fewer items can match or surpass 
the reliabilities and validities of conventional music 
tests. The primary purpose of this study was to assess 
the efficiency, reliability and validity of an adaptive 
music test that used a pool two to three times smaller 
than those employed in previous investigations. 

Advantages of Adaptive Testing 

Commercially-available tests of musical ability are 
inferior to adaptive tests because they use a fixed set of 
items that are administered to all examinees regardless 
of their ability levels. Since examinees generally difia 
in the ability measured by a given test, some items wili 
be too easy and others too difficult to be informative 
about their abilities. Nonetheless, examinees may 
devote considerable time and effort to answering such 
items. Adaptive tests are aimed at improving 
measurement quality by successively presenting items 
of appropriate difficulty, selected from a large item 
pool, to each examinee according to performance on 
previous items. When this procedure is employed, 
examinees respond only to items matched to their 
ability levels, making the test more efficient; they 
respond to a greater proportion of items matched to their 
ability levels, making the test more reliable; and since 
they respond to a more reliable set of items, their scores 
potentially ire more valid. 

Adaptive Testing Research Outside of Music. 
The anticipated advantages of adaptive tests over 
conventional fixed-item tests have been demonstrated in 
studies by McBridc and Martin (1983), Moreno, Wetzel, 
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McBride & Weiss (1^84), Urry (1977), and Weiss 
(1982). Taken as a whole, these studies indicate that 
adaptive testing procedures require an average of 30% 
fewer items to reach reliabilities and validities 
comparable or superior to those of conventional tests. 
These promising results have led to the development of 
the flrst operational adaptive aptitude tests, which 
presently include adaptive versions of the Armed 
Services Vocational Aptitude Battery (Sympson. Weiss 
& Ree, 1982), and the Differential Apil»udc Tests 
(McBride, 1986). 

Adaptive Testing Research in Music. It is 
important to point out that with one exception 
(Vispoel, Coffman and Scriven, 1990), all present 
operational computerized adaptive tests use visually- 
administered items. However, adaptive testing 
procedures need not be restricted to visual stimuli. In 
fact, it can be argued that adaptive tests for assessing 
music listening abilities may be more beneficial than 
adaptive tests assessing skills via visually-administered 
items. The reason for this is that fatigue and lapses in 
concentration have a greater impact on performance on 
listening tests than on visually-administered tests 
(McLeish, 1968). Since listening test items usually are 
presented only once, any lapse in attention may result 
in an incorrect item response. This problem is less 
serious in visually-administered tests, because 
examinees are able to reread a given item as many times 
as they desire before responding. The high level of 
concentration required on listening tests is difficult to 
maintain over a long period of time and quickly leads to 
examinee fatigue. If an examinee experiences fatigue or 
loses concentration during a listening skill test, the 
resulting scores are likely to be invalid. Thus, the 
improvement in test eHiciency gained through adaptive 
testing procedures is particularly desirable when 
assessing musical ability. 

The only research to date on adaptive tests of musical 
ability has been conducted by Vispoel (1986, 1987a, 
1987b, 1990) and his associates (Vispoel and Twing, 
1989; in press). Vispoel (1987a) developed an 
operational version of a test designed to assess tonal 
memory (the ability to remember melodies) on the 
PLATO computer system. Subsequent studies (Vispoel 
and Twing, 1987; Vispoel, 1990) employing computer 
simulation techniques to compare the efficiency, 
reliability, and validity of this adaptive test to a variety 
of conventional music tests yielded results similar to 
those found for adaptive tests using visually- 
administered items. Specifically, the adaptive test, using 
up to 69% fewer items, yielded reliabilities and 
validities comparable to or superior to those of 
conventional music tests. 

While research assessing the feasibility of adaptive tests 
of musical ability has been promising, it is important 
to note that the actual development of an adaptive test is 
time consuming and complicated. Part of the difficulty 
is in constructing an item pool that contains a sufficient 
number of items to measure all examinees ability levels 



with high precision. In previous investigations 
(Vispoel, 1987a, 1990; Vispoel and Twing, 1989), 
pools of 180 and 278 items were used. The goa! of the 
present investigation was to assess the feasibility of 
developing an adaptive test of musical ability with a 
considerably smaller item pool. 

Purpose of the Study 

This study had three main objectives: 

1. To compare the reliability of the adaptive test 
to those of three types of conventional tests 
formed from the adaptive test pool: peaked 
tests, flat or rectangular tests, and tests 
containing the most discriminating items. 
[These comparisons were made at several test 
lengths (5, 10,..., 40 items).] 

2. To assess the average number of adaptive and 
conventional test items required to yield 
selected levels of reliability (i.e., .80, .83, 
.90). 

3. To compare the validity coefficients for the 
adaptive and conventional tests at various 
lengths using several criterion measures 
including the Drake Musical Memory Test, the 
Seashore Tonal Memory Test, and indices of 
musical experience and training. 

Methods and Data Source 

The present study used data from which an operational 
computerized adaptive test of tonal memory was 
developed (See Vispoel, 1987a; Vispoel, Coffman, and 
Scriven, 1990). Items for the adaptive test were 
constructed to parallel those of commercially available 
tests, notably, the Seashore et al. (1960), the Wing 
(1968) and the Bentley (1966) tonal memory tests. A 
typical test item requires an examinee to compare two 
versions of a shoit melody. On the second playing, 
either the same melody is repeated or one note is altered. 
Examinees indicate the number of the note that has 
changed or indicate that the two playings of the melody 
are the same. 

The adaptive test item pool contained 93 items, 
approximately 1/2 to 2/3 fewer items than those used in 
the previous investigations. In addition to responding to 
all of the 93 items in the pool, examinees completed 
the Seashore Tonal Memory Test, the Drake Musical 
Memory Test and an investigator-designed questionnaire 
assessing musical background and training. These latter 
measures, which served as criterion variables in the 
investigation of test score validities, are described in 
more detail in Table 1. 

The present study simulated computerized adaptive tests 
of various lengths using the actual responses of 163 
examinees from the Vispoel (1987a) study. A 
**maximum information** item selection strategy was 
employed (See Hamblcton and Swaminathan, 198S, 
Chapter 7). The first item on each adaptive test is the 
most discriminating item available at average difficulty. 
That item is scored immediately and ability is 
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estimated. Ntxt, the item that provides maximum 
information at the estimated ability level is administered 
and ability is recsiimatcd. The third item is chosen to 
provide maximum information given the new ability 
estimate. This process is continued until the test reaches 
the desired length (5 items, 20 items, etc.). 



Table I. A Description of Criterion Measures. 



MEASURE 


DESCRIPTION 


Drake Musical 
Memory Test 


A 54-iicm standardized 
test of musical memory. 


Seashore Tonal 
Memory Test 


A 30-itcm standardized 
lest of tonal memory. 


Professional Musical 
Performing Experience 


Examinees indicated 
whether they had ever 
played music 
professionally (been paid 
to perform music). They 
were coded 1 if they had 
played professionally and 
0 otherwise. 


College Music 
Experience 


Examinees indicated 
whether they were music 
majors or music minors. 
They were co<Jed 1 if they 
were a music major and 0 
otherwise. 


Insunment Playing 
Experience 


Examinees indicated the 
number of years they have 
played a musical 
instrument. 



In addition to the adaptive test, three types of fixed-item 
conventional tests (flat, peaked and maximum 
discrimination) were developed from items within the 
adaptive test pool. In a fashion similar to McBridc and 
Martin (1983), and Vispoel and Twing (1989), test 
lengths identical to the simulated computerized adaptive 
tests were generated by identifying subsets of items that 
for the flat lest were most discriminating across a wide 
range of abilities, for the peaked test were highly 
discriminating in the middle of the ability range only, 
and for the maximum discrimination test were the most 
discriminating items in the pool independent of 
difficulty level. 

Item parameter estimates, based on data from 468 high 
school and college students, were obtained using 
LOGIST V (Wingersky, Barton, & Lord, 1982) under 
the modified 3-parameter logistic model (i.e., the "c" or 
guessing parameter estimates were fixed). These items 
were calibrated and tested for unidimensionality and 
model fit in a previous study (Vispoel, 1987a). 

Since scores for the present simulated adaptive and 
conventional tests were based on actual responses to test 
and questionnaire items, reliability and validitv results 
should be similar to those that would be obtained if 
examinees had actually taken the conventional and 
adaptive tests in the prescribed manner. All adaptive and 
conventional tests were scored using item response 



theory procedures to eliminate possible confounding of 
results due to scoring method difTercnccs. 

Results 

Reliability. Estimated reliabilities for the adaptive 
and conventional tests at various fixed test lengths are 
given in Table 2. Reliability estimates arc based on a 
formula suggested by Urry (1977) and Green, Bock, 
Humphreys, Linn and Reckase (1984) in which 
reliability is defined as "One minus the reciprocal of test 
information." As can be scw»i from Table 2, the adaptive 
test reliabilities matched or exceeded those of the 
conventional tests at all test lengths. The differences 
between the adaptive and conventional test reliabilities 
were most pronounced at short test lengths (i.e., 5 and 
10 items). 



Table 2. Estimated Reliabilities at Various Test 
Lengths. 





. ypc of Test 


Test 








Max. 


Lenj^th 


Adaptive 


Peaked 


Flat 


Disc. 




.80 


.00 


.00 


.00 


10 


.90 


.62 


.71 


.78 


15 


.92 


.86 


.82 


.87 


20 


.93 


.91 


.87 


.91 


25 


.94 


.94 


.90 


.93 


30 


.95 


.95 


.92 


.95 


35 


.96 


.96 


.92 


.96 


40 


.96 


.96 


.92 


.96 



Efficiency. The average number of adaptive and 
conventional test items required to obtain reliabilities of 
.80, .85, and .90 are given in Table 3. As shown in the 
table, the adaptive test rcach<»d reliabilities of .80, .85, 
and .90 after an average of only 5, 7, and 10 items, 
respectively. In comparison to the conventional tests, 
the adaptive test required from 44% to 62% fewer items 
to reach these reliabilities. 

Validity. Table 4 contains the validity coefficients 
between the criterion measure scores described in Table 
1 and scores from the adaptive and conventional tests at 
selected test lengths. Although less striking than the 
results for efficiency and reliability, it is apparent from 
Table 4 that, in most cases, the adaptive test yielded 
higher validity coefficients than the conventional tests. 
Consistent with the findings for reliability, differences 
in validity favoring the adaptive test were greatest at 
short lengths. The most sizable increases in validity 
coefficients of all tests occurred as test length increased 
from five to ten items. Validity coefficients for all tests 
were reasonably stable after 15 to 20 items. Combining 
the results for the three analyses, it would appear that 
15Mtem adaptive tests provide an excellent combination 
of high efficiency, reliability, and validity. 
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Educational Significance 

The results of this study along with previous 
investigations by Vispoel and his associates strongly 
support the feasibility of adaptive rests of musical 
ability. These tests may help overcome the problems of 
poor reliability and eflTiciency that have plagued even the 
best existing conventional musical ability tests such as 
Gordon's Musical Aptitude Profile. (1968). As 
Whellams (1971, p. 418) notes, "The norms associated 
with Gordon's Musical Aptitude Profile indicate that 
ninety-nine percent of testees in the standardization 
sample scored at least one-quarter to nearly one-half 
(depending on age) of the available points. In other 
words, all testees spend at least a quarter of the total test 
administration time, i.e., about two and a half hours 
responding to items for which play no part in 
discriminating between them. This means that on the 
average, every MAP tester has forty minutes of his time 
wasted." 

The problems that Whellams points out are exactly 
those that are most readily solved by adaptive testing. 
The present results suggest that important 
improvements in reliability, efficiency and validity can 
be realized with adaptive music tests having relatively 
small item pools. 



T able 4. Validity Coefficients Between Adaptive/Conventional Test and Selected Criterion Measure Scores 



Qiterion 


Test 


TcstLenRlh 






5 


10 


15 


20 


25 


30 


35 


40 


Drake Test 


Adaptive 


.63 


.68 


.69 


.72 


.71 


.73 


.72 


.73 


(N=82) 


Peaked 


.57 


.60 


.66 


.66 


.68 


.68 


.69 


.71 


Flat 


.54 


.54 


.64 


.66 


.67 


.70 


.69 


.70 


Max. Disc. 


.59 


.63 


.69 


.67 


.67 


.68 


.68 


.69 


Seashore Test 
(N=82) 


Adaptive 


.68 


.76 


.80 


.80 


.81 


.82 


.83 


.83 


Piaked 


.69 


.73 


.78 


.78 


.79 


.80 


.81 


.81 


Flat 


.62 


.72 


.75 


.74 


.75 


.77 


.77 


.79 


Max. Disc. 


.64 


.72 


.78 


.77 


.78 


.79 


.77 


.77 


Professional 
Performing 
ExpcncKt 
(N.163) 


Adaptive 


.43 


.48 


.48 


.48 


.48 


.49 


.48 


.49 


Piaked 


.37 


.44 


.48 


.48 


.50 


.51 


.50 


.51 


Flat 


.39 


.43 


.45 


.48 


.48 


.48 


.49 


.49 


Max. Disc. 


.39 


.47 


.47 


.49 


.49 


.51 


.51 


.52 


College 
Music 
Experience 
(N=163) 


Adaptive 


.49 


.54 


.55 


.55 


.56 


.56 


.56 


.57 


Piaked 


.42 


.49 


.50 


.52 


.55 


.55 


.55 


.55 


Flat 


.41 


.48 


.52 


.55 


.55 


.55 


.56 


.55 


Max. Disc. 


.42 


.52 


.53 


.55 


.55 


.57 


.57 


.59 


Instrument 
Playing 
Expcrieivx 
(N=163) 


Adaptive 


.54 


.56 


.57 


.57 


.58 


.59 


.59 


.60 


Peaked 


.50 


.55 


.54 




.57 


.57 


.57 


.56 


Flat 


.45 


.51 


.55 


.57 


.58 


.57 


.59 


.59 


Max. Disc. 


.46 


.55 


.57 


.58 


.58 


.59 


.60 


.60 



Table 3. Test Lengths Needed to Reach Selected 
Reliability Levels 





Reliabil 
Level 


ity 


.80 


.85 


.90 


Average Test Length 


Adaptive 


5 


7 


10 


Peaked 


13 


17 


23 


Flat 


12 


14 


18 


Max. Disc. 


11 


13 


18 


Average Difference in 
Test Length 
(Cbnventional- 
AdaDtive^ 


Peaked- 


o 


10 


13 


Flat-Adaptive 


7 


7 


g 


Max Disc • 
Adaptive 


6 


6 


8 


Percent Reduction in 
Test Length 
(Ads^ve versus 
Conventional) 


Peaked vs. 
Adaptive 


62 


59 


57 


Flat vs. 
Adaptive 


58 


50 


44 


Max. Disc, 
vs. Adaptive 


55 


44 


44 
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USING UTAH PILOT TO CONSTRUCT A CAI PROGRAM 
TO REPLACE THE GENERAL LIBRARY TOUR 



V. Lonnie Lawson, Central Missouri State University 



Abstract 

The purpose of this study is ihrcc-fold. First it 
determines if college freshmen who use a computer- 
assisted instruction (CAI) program will have higher 
adjusted post-test means than students who take a 
traditional library tour or students with no formal 
library instruction. Second, it ascertains whether a 
computer-assisted instruction program is more cost 
effective than the traditional library tours. Finally, it 
reveals if students prefer being taught general library 
orientation and the use of basic reference tools using a 
computer-assisted instruction program instead of taking 
a traditional library tour. 

Computer-assisted instruction has been widely used 
throughout various levels of education as a teaching 
tool in lieu of the more traditional methods of lecturing, 
working with programmed texts or using diverse iypes 
of audio-visual instructional methods. Schloss, 
Wisniewski and Cartright, in their 1988 article "The 
Differential Effect of Learner Control and Feedback in 
College Students' Performance on CAI Modules" in the 
Journal of Educational Computing Research, state that 
the research and use of computer-assisted instruction is 
very much of current interest in higher education 
(Schloss, 1988, p. 141); however, little is presently 
being done in the library profession. Even though an 
interest in using computerJ in the academic library has 
been present for some lime, Mignon Adams expressed 
concern in her April 1981 Catholic Library World over 
the lack of instructional media materials for use in 
academic libraries. A<f..Tis noted that numerous 
commercially- produced materials existed for elementary 
and secondary iibraries. She surmised that although few 
commercially- produced materials were being generated 
for academic library use, locally produced media 
materials proved that academic librarians were very 
interested in using this type of instruction CAdams, 
1981, p. 397). 

The purposes of this study were to 1) determine whether 
college freshmen who used a computer-assisted 
instruction program would have higher adjusted po:,i- 
test means than students who took a traditional library 
tour or students with no formal library instruction, 2) 
ascertain whether a computer-assisted instruction 
program is more cost effective than the traditional 
library io>irs, and 3) discover if students prefer being 
taught general library orientation and the use of basic 
reference tools using a computer-assisted instruction 
program to taking a traditional library tour. To 
accomplish these first two objectives, instructors of 
ENG 1020 Rhetorical Wriang classes at Central 
Missouri Slate University v ere a.skcd to volunteer their 



classes to participate in an experimental study. The 
eight classes that volunteered were divided into three 
groups: computer-assisted instruction group, lour group 
and a control group (which received no formal 
instruction). Three classes, with a total of sixty-eight 
students, made up the computer-assisted instruction 
group; three classes, with a total of sixty-four students, 
maw^ up the lour group; and two classes, with forty 
students, comprised the control group. 

Each group was given a pre-iesl and a post-test to 
measure the amount of diffeicnce in the learning which 
took place between the pre- and post-tests. Each test 
also served ar.other purpose. The pre-test was utilized to 
collect demographic data about each student, such as 
current student status, previous involvement in library 
work, knowledge of library skills, Mze of high school 
graduation class, and previous experience or skills with 
computers. The post* test was used to gather 
information on the students' q)inions and evaluation of 
the particular treatment (computer-assisted instruction or 
library tour) to which they were exposed. 

An alpha Ic .el of .05 was set to determine the 
piobabi'iiy level for rejecting cf not rejecting the null 
hypothesis of the first research objective. Analysis of 
Covariance and the SchefTc procedure were the statistical 
methods used to determine the significance of 
relationships between the three groups and among the 
post hoc population sub-groups. The second research 
hypothesis was tested by determining arKl comparing the 
costs needed in order to conduct ENG 1020 tours and the 
CO.SIS necessary for constructing and implementing a 
computer-assisted insu-uction program to provide the 
same instruction. The study was completed by 
comparing the costs of the two teaching methods for 
varying lengths of time - covering one year and for a 
period of five years. 

The final research hypothesis was tested on the basis of 
answers to ten statements on the post-test which asked 
the students to evaluate the method of instruction in 
which they had participated. Two of these statements 
were answered with a "yes" or a "no," seven were 
answered using a Likert-typc scale (with four possible 
choices of strongly agree, agree, disagree, and strongly 
disagree), and the final statement was answered using a 
five-part scale ranging from poor to excellent. 

The researcher, having conducted ENG 1020 tours for 
the past eight years, designed a computer-assisted 
instruction program entitled "Libraries and the Search 
for Knowledge: A Computer Program" using version 
7.0 of Utah PILOT (Programmed, Inquiry, Learning Or 
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Teaching). The make-up of the program follows the 
typical instructional-design procedures and includes: 

1 . An intended audience to use the program. 

2. Identification of the objectives to be covered. 

3. Organization of instruction into specific 
sequences. 

4 . Development cf program content 

5. Procediure to determine the effectiveness of the 
program in covering the objectives (CrandalK 
1987» p. XV). 

Although PILOT is limited wilh regard to graphic 
capabilities, it was chosen because it can be mastered by 
people who do not have a strong computer 
programming background. In essence, a person who 
uses PILOT to consuiict a CAI program needs to be a 
specialist in a particular subject rather than a computer 
programmer. If the experimotnt proves successful, the 
duplication of such a program using PILOT is feasible 
and can be adapted by other librarians for use at any 
academic institution. 

The ENG 1020 computer-assisted instruction program 
was constructed in separate modules, totaling thirteen. 
In Older to begin using the progiam the user places the 
disk in the computer drive and turns on the machine. 
The program is equipped with an autoexec.bat file that 
automatically takes the user to the beginning of the 
program. Once underway, the student must complete the 
program in a single session. The following title of each 
module follows. 

Module One: Introduction to the Program. 
Module Two: User Informat>on/Interacuon. 
Module Three: Introducticii to Ward Edwards Library. 
Module Four Dewcy/LC Oassificalion Systems. 
Module Six: Library of Congress Subject Headings 
Guide. 

Module Severn The Catatog Tard 

Module Eight: Library of Congress Call Number. 

Module Nine: Location of Books. 

Module Ten: PeriodicalsyPeriodical Indexes. 

Module Eleven: Readers* Guide to Periodical Literature. 

Module Twelve: Newspapers. Module Thirteen: Final 

Exam. 

Summary of Findings 

The present study confirmed earlier research which had 
reported that the vast majority of students utilizing 
computer-assisted instruction learned as much as, or 
more than, those students using more traditional 
methods of instruction. An analysis of findings for the 
teaching method effectiveness part of this study is 
discussed first, followed by the findings of the students* 
evaluation of the methods of instruction, the cost 
effectiveness results and the post hoc analyses. 

Teaching Method Effectiveness. At the onset the 
three groups were comparable in the scored mean 
averages after taking the pre-test. After each group 
concluded the post-test, the computer-assisted 
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insuiiction group showed a signiflcantly higher adjusted 
post-test mean, achieving a probability level of .0001. 
This ft.: exceeded the alpha level of .05 that was 
originally set Another set of analyses was conducted to 
determine if there were individual differences between 
the computer- assisted instruction group*s adjusted post- 
test means and those of the tour group and the control 
group. The results of the analysis of the individual 
adjusted post-test means showed that the computer- 
assisted instruction group, when compared to the tour 
group and the control group, was shown to be 
significantly different at the .05 level. The comparison 
of the adjusted post-test means of the tour group also 
indicates that this teaching method proved to be 
significantly different at the .05 level over the adjusted 
post-test means of the control group. 

Teaching Method Evaluation. Responses to the 
ten questions on the post-test designed for collecting 
data for evaluating the students* opinions of the teaching 
method to which they were exposed were tabulated. 
Most students in both the CAI group and the tour group 
felt that their respective teaching method helped them to 
better understand the lay-out of the library and the basic 
research tools located there. The majority of students in 
each group also felt that the material was adequately 
covered in the CAI program or by the tour guide. There 
was also little disagreement among the two groups of 
students when asked if they would recommend that other 
students select the treatment to which they were 
exposed. Both groups would recommend the leaching 
method to other suidents. When asked about their 
ability to take notes while e^riencing the treatment, a 
high njajority (88.2%) of the computer-assisted 
instructior group indicated they were allowed tir^e to 
take notes, with sixteen (23.5%) stating that they did 
take notes. Oly thirty-five students (54.7%) in the 
tour group felt they had time to t^e notes, with two 
students (3.1%) actually doing so. 

Another area of disagreement between the two groups 
sqppeared when they were asked if the treatment had been 
too long. Even though the time for both treaunents was 
virtually the same, less L*ian fifty percent (45.3%) of the 
tour group felt the tour was too long, while over eighty 
percent (82.4) of the CAI group felt the treatment was 
too long. This does not seem necessarily one-sidec* 
when one considers that the CAI group had to begin and 
end the treaunent in one sitting with no time for a 
l^eak. This factor could accc int for the consensus of the 
CAI group indicating that the CAI program was too 
long. 

Finally, there was little difference in the over-all rating 
given the treatment method by either group. The 
majority in both groups indicated that they felt the 
teaching method in which they had participated was 
average or above. 

Cost Effectiveness Evaluation. Direct costs were 
figured for providing tours and for the construction of 
the CAI program. The tour direct costs, included the 
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librarian's time and the inslruciors time. The direct 
costs for the CAI program were attributed to price of the 
floppy disks, price of software and manual, cost of 
computer equipment, cost of programmer per hour, cost 
of lab assistant and maintenance of equipment. 

Thr*^ models were developed from which to draw 
aiialyses. Each model featured the same projected costs 
for providing, tours, which included the librarian's and 
leaching faculty's time. The differences in the projected 
costs in each model appeared when designing the costs 
for producing and maintaining the CAI program. For 
example, the fust model's CAI costs did not include the 
purchase of computer equipment or a service agreement 
The second models' CAi costs did not include the 
purchase of computer equipment The third model's CAI 
costs included everything previously indicated in 
addition to the purchase of computer equipment 

The cost models' projected outcome shows that there is 
an s^arent difference in the cost comparison of the CAI 
program and ihe tour in the first t^o models from the 



very beginning. However, wfih the addition of the 
purchase of computer equiprffent the third model does 
not prove to be cost effective until several years have 
elapsed. This is due to the cost of the CAI progfam 
remaining virtually stable after the initial cost of the 
purchase of the equipment; while ^e*cost fcH* providing 
tours increases significantly each year. 

t • 
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COMPUTER ANXIETY AMONG TEACHERS 
IN A COMPUTER LITERACY COURSE 



Marcia B. Cushall, Frostburg State University; and Francis A. Harvey, Lp.hijh University 



Abstract 

The purpose of the study was to examine the 
relationships among teacher anxiety, achievement, and 
sequence of topics in an introductory computer literacy 
course. The subjects were elementary and secondary 
teachers who voluntarily enrolled in a university 
computer literacy course. The subjects were randomly 
assigned to two groups; there were 19 subjects in Group 
A and 22 subjects in Group B. The two groups of 
subjects were exposed to each of three computer-related 
topics. The sequence of the topics was varied in the two 
groups. The sequence of topics for Group A was 1) use 
of tool software, 2) use and evaluation of Computer 
Assisted Instruction (CAI), and 3) programming in 
BASIC. For Group B the second and third topics were 
reversed in order. 

State anxiety was measured by the State Trait Anxiety 
Inventory six times during the study. To measure 
cognitive achievement, a cognitive test containing 10 
questions on each of the 3 topics was administered as a 
pretest, posttest. Perceived value of hands-on 
experiences was measured three times, once for each of 
the three topics, by a Topic Questionnaire. 

Results of statistical analyses indicated that: 

there was a significant relationship between state 
anxiety during specific topic instruction and the 
sequence of tq)ics; 

• the overall decrease in state anxiety for Group B 
was significantly greater than the overall decrease 
for Group A; 

here was no significant relationship between overall 
state anxiety change and overall cognitive 
achievement; 

only the negative correlation between perceived 
value of training in the use of CAI and anxiety 
level for Group A was significant; and, 

• there was no significant relationship between 
cognitive achievement and the sequence of topics. 

Of the two topic sequences studied, the sequence in 
which programming in BASIC was taught as the second 
topic was better for reducing computer anxiety. There 
was no optimal sequence for greater cognitive 
achievement There was no general relationship between 
anxiety level and perceived value of training. More 
research must be conducted using all possible variations 
of topic sequence lo determine if there is an optimal 
sequence for greater reduction in anxiety and greater 
cognitive achievement 



Introduction 

In order for the computer to play a greater role in 
preparing students to live and compete in this 
technological age, schools must adapt with instructional 
changes. There is "almost unanimous consensus that 
things must change. Most educators believe that the 
new tools of the information age can be pivotal in 
shaping the American classroom to fit its ever-changir^ 
environment" (Power On!, 1988, p. 201). 

In order for computer technology to have an impact on 
learning, teachers must be comfcHtable with computers 
and see them as tools that enhance daily teaching. For 
this, teachers need training (Power On!, 1988). More 
than eight years after the first micrc<:omputers started 
being used in the schools, teacher training continues to 
be one of the most crucial components to the success of 
any educational technology program (Chaffm, 1988). 
As the Alaska Department of Education points out in 
Electronic Learning's 1988 Survey of the States, 
"teacher training is necessary to implement any major 
instructional change" (Chaffm, 1988, p. 45). 

Although state support for teacher inservice training in 
the use of computers has been increasing over the years 
(Bruder, 1988), the majority of teachers have received 
little training. The report of the Office of Technology 
Assessment (Power On!, 1988) indicates that only one- 
third of all K-12 teachers have 10 hours of inservice 
computer training. The summary of findings from the 
NAEP report "Computer Competence: The First 
National Assessment** indicates that most teachers 
graduated from college when few computer courses were 
offered^While this lack of formal training in computers 
does not necessarily disqualify them from performing 
well, too many teachers make low evaluations of their 
knowledge and skill" (Bracey, 1988b). 

The momentum for computer use in the classroom and 
inservice training for their use has increased (Bracey, 
1988a; Bruder, 1988), but there still remains a certain 
amount of anxiety about using them"Indeed, there have 
been times when I thought that the kids were catching 
anxiety from teachers" (Bracey, 1988, p. 20). The Office 
of Technology Assessment reports that one of the 
barriers to use of computers in the classroom is 
intellectual - initial fears of using the computer (Power 
On!, 1988). The results of an administrative, survey 
conducted by Electronic Learning and reported by 
Andrew Barbour (1987) indicated that seventeen percent 
of administrators surveyed cite fear of computers as a 
major problem facing wider implementation of 
technology in their school or district (Barbour, 1987). 
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The report of the Office of Technology Assessment 
(Power Ox!, 1988) notes that in order for teachers to 
feci in cortrol, initial fears about using computers may 
need to be overcome. Once teachers can see that the 
computer is a tool that can help them do their job and 
once they fe^l comfortable in using the computer, they 
will try to integrate it into i^eir classes. Inscrvice 
training must help teachers overcome their anxieties 
about using computers. 

Inservice training in technology must be sensitive to 
the concems or anxieties with which teachers approach 
the use of technology. Many teachers, especially those 
who consider themselves "B.C." (before computers) 
have not yet worked with computers and admit to being 
"technophobic." Others had early negative computer 
training experiences. Sometimes programming was 
emphasized; sometimes the courses tried to cover too 
much, too fast and have no relevance to their teaching 
needs. Several factors contribute to a teacher's anxiety 
about computers; they must be taken seriously as they 
underlie whether or not a teacher adopts technology. 
{Power On!, 1988, p. 104) 

Anxiety is the result of psychological stress. The 
consequences of anxiety dep^ ' on the demands of the 
situation and are usually negative and debilitating. 
Anxiety induced by interest and excitement is the main 
facilitator of constructive behavior and creative living 
(Izard & Tomkins, 1966). If a person does not 
experience a certain amount of anxiety, he is likely to 
be ineffective (Grinker, 1966). But a rise in anxiety 
level induced by a perceived threat or by the complexity 
of a situation produces disturbances in performance and 
inefficiency (Grinker, 1966). Anxiety has a tendency to 
feed upon itself and become more intense. As a person's 
ineffectiveness becomes clearer, anxiety mounts 
(Grinker, 1966). 

If teachers are to use computers effectively in the 
classroom, they must fu^t overcome their anxiety about 
using the computer. To help teachers overcome this 
anxiety, one must understand what factors contribute to 
this anxiety and what experiences will alleviate the 
anxiety. These factors must be taken into account when 
planning any program for teacher training in the use of 
computers {Power On!, 1988). 

How does subject matter taught in inservice programs 
affect anxiety? Computer anxiety is situational. 
Reducing anxiety in one context does not imply that it 
will not reoccur in another context. If there arc changes 
in anxiety levels brought about by changes in subject 
matter, anxiety-elevating experiences could be 
eliminated or, if they must be presented, adjustments 
could be made in the way they are taught to take into 
account the anxiety they will produce. 

When confronted with a new experience, teachers think 
first about how it affects them (Bracey, 1988a). 
Researchers found that teachers in beginning stages of 
learning had mainly self-centered concerns. The Office 



of Technology Assessment report {Power On!, 1988) 
indicated that instructors for computer inservice training 
must keep in mind teachers' special concems regarding 
computers when they are planning inservice activities. 
Is there a relationship between anxiety levels and 
teachers' concems about the value of the inservice 
computer training? 

The study reported in this paper investigated the 
following questions: 

1. Is there a difference in teachers' levels of 
computer anxiety due to sequence of hands-on 
computer experiences? 

2. Is there a difference in teachers' overall 
cognitive achievement in computer-related 
topics due to their levels of computer anxiety? 

3. Is there a relationship between teachers' 
computer anxiety and their perceptions of the 
value of the inservice training to them? 

4. Is there a difference in teachers' achievement in 
computer-related topics due to sequence of 
hands-on computer experiences? 

Review of the Literature 

Research studies involving computer anxiety have 
investigated relationships between compute anxiety and 
exposure to computers, age, sex, prior experience, and 
mathematics anxiety . 

Exposure to computers. Most studies involving 
computer anxi;^ty are in agreement that anxiety is 
reduced with exposure to computers. A typical example 
is the Honeyman and White (1987) study in which 38 
adults enrolled in computer literacy classes were 
administered the State Trait Anxiety Inventory (STAI) 
as a pretest, posttest, and midterm test. Results 
indicated that anxiety levels changed significantly after 
30 hours of contact with the compute. 

The exposure or "hands-on" experience given to the 
subjects in the studies includes computer programming, 
problem solving using commercially produced computer 
programs, use of tool software, and examination of 
instructional software. Usually the experience is of one 
type and not a variety. Computer anxiety is situational 
so that reducing anxiety in one context does not imply 
that it will be reduced in another context. The studies 
cited do not address the question of how anxiety levels 
might change as tc^ic areas change. 

Age. Most studies have found no correlation between 
age and computer anxiety. A representative study is the 
study by Raub (1981) who found no linear relationship 
between computer anxiety and age of undergraduate 
students enrolled in courses involving instructional use 
of computers or introductory COBOL. 

Results from some of the studies cited must be 
examined with care. In the study by Raub (1981), for 
example, the range in ages for subjects was only 17 
years to 21 years. It is questionable whether or not this 
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range in age is great enough lo determine if there is a 
correlation between age and computer anxiety. 

Sex. The relationship between computer anxiety and 
sex has been reported with mixed results. For example, 
Honcyman and White (1987) in their study of change in 
computer anxiety over time found sex to have no 
significant correlation with computer anxiety. Jordan 
and Stoup (1982), however, in a study relating self- 
reports of fear to several variables, found a significant 
correlation between sex and computer anxiety. 

Results of some of the studies examining the 
relationship between sex and computer anxiety should 
be examined more closely. In the study by Jordan and 
Stroup (1982), no data were given on the number of 
male and female students enrolled in the experimental 
group. Subjects in the experimental group were enrolled 
in a data analysis course in which computers were used. 
There is a possibility that more men than women were 
likely to enroll in a data analysis course during that 
time period. If the subjects voluntarily enrolled in this 
course in which computers were being used, it is 
possible that they were less anxious about using a 
computer than students who enrolled in courses in 
which computers were not used. 

Prior experience. Most studies report a negative 
correlation between computer anxiety and prior 
experience. The subjects with less computer experience 
were generally more anxious. A representative study is 
the study by Jones and Wall (1985) which found a 
negative correlation between anxiety scores and 
computer experience of 127 undergraduate students 
enrolled in the same computer class. 

Math anxiety. Only a few studies have included 
measurements of math anxiety but those studies do 
indicate a correlation between math anxiety and 
computer anxiety. For example, Gressard and Loyd 
(1987) conducted a study to investigate the effects of 
mathematics anxiety and sex on ccn^puter anxiety using 
356 junior high through college aged students. Results 
indicated a significant amount of variance of computer 
anxiety was explained by inclusion of mathematics 
anxiety in the regression equation. 

Additional variables. Very few studies have 
included measurements of cognitive achievement. One 
study that did examine this variable was conducted by 
Marcoulides (1988). The subjects were enrolled in a 
computer information systems course. Cognitive 
achievement was measured on grades of course 
assignments. Computer anxiety accounted for 53% of 
variation in achievement. If the assignments were 
project or programming type assignments, the 
objectivity in grading the assignments could be 
questioned. The location in which the assignments were 
completed could also be questioned. If assignments were 
completed at home rather than in the classroom, 
subjects might have received help with the assignments. 



For some of the studies, valid and reliable instruments 
such as the STAI and CAIN were used. For many 
studies (Jordan & Stroup, 1982; Lindbeck & Dambrot, 
1986; Raub, 1981; Dambrot, Watking-Malek, Silling. 
Marshal, & Garver, 1985), however, anxiety 
instruments were created specifically for the study. 
Those instruments might have addressed the questions 
PsOsed in the study, but the reliability of the instruments 
car. be questioned. 

More evaluative research is needed to determine: 1) 
whether computer anxiety is affected by type of hands- 
on experiences with computers and if so, how it is 
affected; 2) whether cognitive achievement is affected by 
level of computer anxiety; and 3) whether correlations 
exist between age and computer anxiety, and between 
sex and computer anxiety. 

Design of the Study 

The population sampled for this study was one of 
convenience. The subjects were elementary and 
secondary teachers who voluntarily enrolled in the 
Microcomputers in Education courses at Lehigh 
University for the Fall 1989 semester. This sample was 
selected because the majority of subjects were computer 
novices with little or no previous computer experience. 
The literature on computer anxiety suggested that 
subjects with little or no computer experience can be 
expected to exhibit a high level of computer anxiety. 

The initial sample for the study consisted of 46 K'12 
teachers. The subjects were randomly assigned to two 
groups. Group A and Group B, each containing 23 
subjects. In each group there was one person assigned to 
each computer allowing for maximum hands-on time 
for each person. Five subjects withdrew from the 
literacy course, four within the first two weeks, so that 
the final sample contained 41 teachers, 19 in Group A 
and 22 in Group B. In Group A 31.6% of the subjects 
were male, 68.4% of the subjects were elementary 
teachers and 15.8% had never used a computer prior to 
taking the ITEC course. In Group B 18.2% of the 
subjects were male, 68.2% of the subjects were 
elementary teachers 40.9% had never used a computer 
prior to taking the ITEC course. 

The same instructor taught both groups to control for 
error due to personality and difference in teaching style 
of instructOTS. The instructor selected to teach the course 
to the two groups had taught the course for eight 
semesters and had always received excellent ratings as 
indicated by the course evaluation forms which were 
completed by the students at the completion of the 
course. 

Three major topics were taught by the instructor. They 
were 1) use of tool software, 2) programming in 
BASIC, and 3) evaluation and use of Computer Assisted 
Instruction (CAI). Although some time was devoted to 
other activities ' ich as mandatory testing, discussion of 
computer ethics, and proposal writing, the majority of 
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time was spent on activities relating to the three major 
topics. Four consecutive weeks were devoted to the use 
of tool software, three consecutive weeks to evaluation 
and use of instructional software, and three consecutive 
weeks to programming in BASIC. 

The sequencing of these three topics was varied in the 
two groups. Although there are six possible sequences 
of topics, there were only two groups of subjects 
available for this study. Therefore, the two sequences of 
topics which provided clearest contrast in terms of 
placement of program components were studied. The 
sequence of topics for Group A was I) use of tool 
software, 2) use and evaluation of CAI, and 3) 
programming in BASIC. The sequence of topics for 
Group B was 1) use of tool software, 2) programming 
in BASIC, and 3) use and evaluation of CAI. 

Detailed course outlines developed by the instructor 
were given to the subjects at the first class meeting. 
The outlines varied only in sequencing of topics. The 
subjects were told what was required of them at t*'e start 
of the course, thus avoiding the chance for a change in 
anxiety level at any testing interval due to anticipation 
of unknown assignments. Any subject withdrawing 
from the course was asked to submit a report explaining 
the reason for the withdrawal. 

Subjects were assigned identification numbers randomly 
by the researcher. This assigned number were used on 
all tests and forms completed by the subjects. The 
master list of assigned numbers was maintained by the 
researcher and was not made available to the instructor. 
A subject forgetting his assigned number could get that 
number from the researcher at the beginning of the class 
period 

Course orientation sessions were held for both groups 
one week prior to the start of the courses in an 
environment which contained no computers. Attendance 
at the orientation sessions was required. At the course 
orientation sessions subjects were given their assigned 
identification numbers and the Human Subjects form, 
the State Trait Anxiety Inventory, and the Subject 
Questionnaire were administered to each subject 

The Slate Trait Anxiety Inventory was administered to 
all students at the start of the first class period and after 
one hour had elapsed in the twelfth class period. The 
State portion of the inventory and the Topic 
Questionnaire were administered to all students after one 
hour had elapsed in the third, seventh, and tenth class 
periods. The inventory was administered only six times 
to all students to conuol for test/retest error. When 
subjects are given the same test repeatedly at close 
intervals of time, they tend to mark the same answers 
(Sax, 1980). 

The STAI was administered by the instructor instead of 
the researcher to conuol for possible bias of the 
researcher. At the start of the first class period, the 
ITEC cognitive test with additional questions relating to 



th^ major topics, the STAI, and the ITEC attitude 
survey were administered as pretests. Subjects were 
asked to write their assigned numbers on all tests and 
forms completed that were related to this study. All 
forms and tests were placed in designated folders by the 
students and placed in the researcher's office at the end of 
the class period by the instructor. The instructor did not 
see any of the forms and tests completed by the students 
except for the course evaluation forms. Identifying 
numbers were removed from the course evaluation 
forms by the researcher before being given to the 
instructor. 

Prior to administering the STAI during the first class 
meeting, the instructor read the original directions while 
the subjects silently followed along. The instructor then 
reminded the subjects to relate all questions in the state 
portion of the inventory to the use of computers. Also 
during the first class a general overview of the course 
contents was presented. The second through fifth classes 
were used to instruct subjects in Groups A and B in the 
use of tool software. After one hour of the third class 
period had elapsed, the state anxiety portioi of the STAI 
was administered by the instructor to all the subjects. 
The instructor read the original directions while the 
subjects followed along. The instructor then reminded 
the subjects to relate all questions in the state portion of 
the inventory to the use of computers for tool software. 

During the sixth through eighth classes, subjects in 
Group A received instruction in the evaluation and use 
of CAI; subjects in Group B received insuuction in 
programming in BASIC. After one hour of the seventh 
class period had elapsed, the state anxiety portion of the 
STAI was administered by the instructor to all the 
subjects. Prior to administering the inventory the 
instructor read the original directions while the subjects 
followed along. The instructor reminded subjects in 
Group A to relate all questions in the state portion of 
the inventory to the use of computers for evaluation and 
use of CAI. The instructor reminded subjects in Group 
B to relate rll questions in the state portion of the 
inventory to ihe use of computers for programming in 
BASIC. 

The ninth through eleventh classes were used to instruct 
Group A subjects in programming in BASIC and Group 
B subjects in the evaluation and use of CAI. After one 
hour of the tenth class period had elapsed, the state 
anxiety portion of the STAI was administered to all 
subjects. Prior to administering the inventory the 
instructor read the original directions while the subjects 
followed along. The instructor reminded the subjects in 
Group A to relate all questions in the state portion of 
the inventory to the use of computers for programming 
in BASIC. The instructor reminded subjects in Group B 
to relate all questions in the state portion of the 
inventory to use of computers for evaluation and use of 
CAI. After one hour of the twelfth class had elapsed, 
the STAI was administered to all subjects as a posttesL 
Prior to administering the STAI during the twelfth class 
meeting, the instructor read the original directions while 
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the subjects followed along. The instructor reminded the 
subjects to relate all questions in the state portion of the 
inventory to the use of computers. 

After taking the state anxiety portion of the STAI 
during weeks 3, 7, and 10, the subjects were asked to 
complete the Topic Questionnaire, relating the 
questionnaire to the topics being taught during those 
weeks. 

During the thirteenth class, the ITEC cognitive test 
with additional questions relating to the major topics, 
and the ITEC attitude survey were administered as 
posttests. The ITEC course evaluation form was also 
administered to the subjects during the thirteenth class. 

During the third, seventh, and tenth class periods for 
each group, the instructor was audioiaped. The tapes for 
both groups were compared to determine if the amount 
of time spent on a topic and the information taught 
were approximately the same for each group. A time 
table of activities is given above. 

Results 

A repeated measures ANOVA was performed to 
determine whether there was a topic sequence effect on 
state anxiety. Results indicated that there was a 
significant relationship between state anxiety during 
specific topic instrucfion and the sequence of topics. 
Results of the analysis are summarized in Table 1. 

The pattern of changes in anxiety levels differed for the 
two sequences. For the time period in which Group B 
subjects changed from programming in BASIC to 
evaluating CAI, subjects displayed a significant decrease 
in state anxiety; during that same time period, subjects 
in Group A changed from evaluating CAI to 
programming in BASIC and displayed a significant 
increase in state anxiety. A graphic comparison of the 
mean slate anxiety scores for each testing period is 
presented in Figure I. A one-way ANOVA was 
performed to determine whether there were significant 
differences in overall state anxiety change scores 
between groups. Significant differences were found 
between the groups in overall change in state anxiety 
from before the start of the course to -veek twelve of the 
course and from week one of the course to week twelve 
of the course. Results of the analyses are summarized in 
Tables 2 and 3. 



Week Group A Group B 



0 


Orientation session 


Saine 




Human Subjects form 






completed, identification 






numbers randomly 






assigned, STAI(S&T) 






and subject information 






questionnaire completed. 




1 


Cognitive pretest (with 


Same 




additional topic items) 






and attitude pretest 






mandated by the state 






completed, STAI (S&T) 






completed 




2 


Instruction - use of tool 


Same 




software. 




3 


STAI(S) and topic 


Same 




questionnaire completed. 






instruction - use of tool 






software. 




4 


Instruction - use of tool 


Same 




software. 




5 


Instruction - use of tool 


Same 




software. 




6 


Instruction - evaluation 


Instruction - fHogram- 




and use of CAI. 


ming in BASIC. 


7 


STAI(S) and topic 


STAI(S) and topic 




questionnaire completed. 


questionnaire 




Instruction - evaluation 


completed. Instruction 




and use of CAI. 


- programming in 






BASIC. 


8 


Group A: instruction - 


Instruction - 




evaluation and use of 


programming in 




CAI. 


BASIC. 


9 


Instruction - 


Instruction - 




programming in BASIC. 


Evaluation and use of 






CAI. 


10 


STAI(S) and topic 


STAI(S) and topic 




questionnaire completed. 


questionnaire 




Instruction - 


completed. Instruction 




programming in BASIC. 


- evaluation and use of 






CAI. 


1 1 


Instruction - 


Instruction -evaluation 




programming in BASIC. 


and use of CAI. 


12 


TAI(S&T) completed. 


Same 




instruction - Dther 






topics. 




13 


Cognitive posttest (with 


Same 




additional topic items), 






attitude posttest, and 






course evaluation form 






mandated by the state 






completed. 
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Table I. Two-Factor Reocated 


Measures ANOVA — State Anxiety 


Source 


df 


Sum of Squares 


Mean Square 


F 




GrouDsfA) 


1 


1364.22 


1364.22 


3.171 


.0828 


Subiects within Kroups 


39 


16780.44 


430.27 






Repeated Measure CB) 


5 


11849.45 


2369.89 


24.876 


.0001 


AB 


5 


4665.21 


933.04 


9.794 


.0001 


B X Subjects within Kroups 


195 


18577.01 


95.27 







80 y 

70-- 



60.. 



Mean STAI (S) 
Scores 




20 .| 1 1 \ 1 1 

IWeek 1st 3rd 7th 10th 12th 
Prior Class Class Class Class Class 



STAI Testing Classes 

Figure 1. Mean State Anxiety Scores for Each Testing Period. (Range of possible scores is 20 to 80.) 



Table 2. ANOVA -- State Anxiety Score Prior to First Class to Twelfth Class 



Source 


df 


Sum of Squares 


Mean Square 


F 


P 


Between Kioups 


1 


777.79 


777.79 


5.104 


.0295 


Within Kroups 


39 


5943.72 


152.40 






Total 


40 


6721.5 









Table 3> ANOVA - State Anxiety Score First Class to Twelfth Class 



Source 


df 


Sum of Squares 


Mean Square 


F 


P 


Between Kroups 


1 


684.88 


684.88 


4.527 


.0397 


Within Kroups 


39 


5899.61 


151.27 






Total 


40 


6584.49 









In both instances both groups displayed a decrease in state anxiety. However, the decrease in state anxiety for (Jroup 
B was significantly greater than the decrease for Group A. 

An ANCOVA was performed to determine whether overall change in state anxiety had an effect on overall cognitive 
achievement for topic sequences. Results did not indicate a significant relationship between overall state anxiety 
change and overall cognitive achievement from either before the start of the course to week twelve of the course or 
from week one of the course to week twelve of the course. Results of the analyses are summarized in Tables 4 and 5. 



Table 4. One-Factor ANCOVA— Overall Change in State Anxiety from One Week Prior 
to Start of Course to Week Twelve and Overall Cognitive Achievement for Topic 
S equences ^ 



Source 


df 


Sum of Squares 


Mean Square 


F 


P 


Group 


1 


5.22 


5.22 


.458 


.5026 


STAI change score 


1 


.08 


.08 


.007 


.9319 


Residual 


38 


432.98 


11.39 
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Table One-Factor ANCOV A— Overall Change in State Anxiety from Week One to 
Week Twelve of Course and Overall Cognitive Achievement for Topic Sequences 



Source 


df 


Sum of Squares 


Mean Square 


F 


P 


Group 


1 


3.63 


3.63 


.319 


.5756 


STAI change score 


1 


.88 


.88 


.077 


.7826 


Residual 


38 


432.18 


11.37 







Correlation coefficients were calculated to determine the kind and strength of the relationships between anxiety level 
and perceived value of training. At the .01 level of significance, only the negative correlation between perceived 
value of training in the use of CAI and anxiety level for Group A was significant Results of the analysis are 
summarized in Table 6. 



Table 6. Correlation Coefficients — State Anxiety and T opic Value Quotient 





Group A 


Group B 


Topic 


r 


r2 


r 


r2 


BASIC 


-.447 


.200 


.159 


.025 


CAI 


-.623* 


.388* 


-.084 


.007 


Tools 


-.209 


.044 


-.059 


.003 



A repealed measures ANOV A was performed to determine whether there was a topic sequence effect on achievement 
in each topic ar^a. Results indicated no significant relationship between cognitive achievement and the sequence of 
topics. Results of the analysis are summarized in Table 7. 



Table 7. Two-Factor Repeated Measures ANOV A— Cognitive Achievement by Order of 
Topic ^ 



Source 


df 


Sum of Squares 


Nfean Square 


F 


P 


Groups (A) 


1 


1.81 


1.81 


.489 


.4886 


Subjects within groups 


39 


144.35 


3.70 






Repeated Measure (B) 


2 


8.31 


4.15 


1.87 


.1609 


AB 


2 


8.43 


4.22 


1.90 


.1568 


B X subjects within ktoups 


78 


173.26 


2.22 







Of the two topic sequences studied, the sequence in which programming in BASIC was taught as the second topic 
was belter for reducing computer anxiety than the sequence in which programming in BASIC was taught as the third 
topic. There was no optimal sequence for greater cognitive achievement There was no general relationship between 
anxiety level and perceived value of training. More research must be conducted using all possible variations of topic 
fr^uence to determine if there is an optimal sequence for greater reduction in anxiety and greater cognitive 
fA:hievemenL 
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INSTRUCTIONAL GROUPWARE: DESIGN CONSIDERATIONS 



Norman R. Dodi Virginia Tech 



The thesis of this paper is rooted in the literature on 
cooperative learning, group use of computers in 
instruction and principles of instructional design of 
educational software. The thesis: that instructional 
software must be designed to specifications suggested 
by principles of cooperative learning if the computer is 
to reach its potential as a tool to facilitate learning in 
small groups. What follows is a case for the importance 
of the smdl group in classroom instruction, a critique 
of the research on group use of computers in 
instruction, a derived set of principles of cooperative 
learning and a frameworic for the design of computer 
software which is optimally suited for facilitating small 
group instructional practice. 

A quotation from Joyce and Weil (1986) sets the stage 
for consideration of the importance of cooperative 
learning: 

Classrooms organized so that students work in 
pairs and larger groups, tutor each other and share 
rewards are characterized by greater mastery of 
materials than the common individual-study-cum- 
recitation pattern. Also, shared responsibility and 
interaction produce more positive feelings toward 
tasks and toward others, generate better intergroup 
relations, and result in better self-images for 
students with histories of poor achievements (p. 
216). 

Joyce and Weil draw these conclusions based on their 
own research (Bavja, Showers, aiid Joyce, 1985 cited in 
Joyce & Weil, 1986) and on that of others who have 
investigated what has come to be labeled cooperative 
learning (Johnson and Johnson. 1984; Robert Slavin, 
1983; Shiomo Sharan, 1980). 

Cooperative learning practices are those instructional 
strategies which depend on the interaction of a small 
group of learners as a central feature of classroom 
learning tasks. A small group is a ''collection of 
interacting persons with some degree of reciprocal 
influence over one another" (Schmuck & Schmuck, 
1988, p. 20). These persons interact or collaborate with 
one another and form a consensus of ideas that 
represents the group as a whole. Specific to education, 
Bruffcc (1984) stated that the purpose of groups in a 
collaborative learning environment is "to create and 
maintain a demanding academic environment that makes 
collaboration-social engagement in intellectual 
pursuits— a genuine part of students' educational 
development" (p. 652). As far as Schmuck and 
Schmuck and Bniffce were concerned, a group should be 
a fun^lamental part of learning in most classrooms. 

Conventional approaches to instruction which make 
intentional use of small groups assume face-to-face 



interaction among students with the teacher serving as a 
counselor, consultant, critic and one who engineers the 
discrepant event which motivates group inquiry. Several 
teaching models presented by Joyce and Weil (1986) arc 
promulgated to promote cooperative learning, ranging 
from social models such as Group Invesugation based 
on the theories of Thelan (1967) and information 
processing models such as Concept Attainment and 
Inductive Thinking. 

What is not considered by Joyce and Weil (1986) is an 
integral role for the computer in the smaM group 
process. Conventional wisdom projects jic of 
computers in individualized learning contexts. Yet, there 
is a fairly extensive literature which supports the notion 
that group use of computers in the classroom may be a 
common instructional practice. 

Research on Groups Using Computers 

Webb conducted numerous studies on small group 
interaction, initially without considering computer use 
as a condition. She concluded that: 

1. An individual in a group is influenced by the 
group (1982c), 

2. Students learned from their peers (1982b), 

3. There was no significant difference between 
mixed-ability groups and uniform groups 
(which refuted some earlier studies) (1982a), 

4. Students' participation and average frequency of 
participation within a group tended to remain 
stable over time (Webb A, Cullian, 1983), 

5. Student and group behavior, in contrast, tend 
to be unstable over time (1984b), and 

6. Verbal interaction (giving and receiving 
explanations) promoted learning in small 
groups if teachers controlled group activities 
(1985b). 

These studies reinforced the theory that small group 
work is beneficial. But, when Webb studied small 
groups rsing computers, the results were not always 
consistent Webb (1984a) tried to analyze the "cognitive 
abilities, cognitive styles, and student demographic 
character: 'Sties" that make up cooperative group learning 
(p. 1076). \hc studied 35 male arid female students from 
ages 11-L years who were learning to use LOGO 
programming techniques and analyzed the group 
processes that occurred during one week. Using a 
stepwi.se, multiple regression analysis for student 
ability characteristics and a partial correlation measure 
between group interaction variables and computer 
programming, she found ihat students working in small 
groups were successful in learning to use LOGO and 
that the group-process variables as well as student 
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characteristics and previous experiences, were predictors 
of programming performance (Webb, 1984a), 

In another study, Webb (1985a) focused on the 
cognitive requirements of learning computer 
programming with groups and with individuals. She had 
30 male and female studcnl^ aged 1 1-14 years participate 
in a ten-lesson workshop lemming to program in 
BASIC. Using a t-test, an analysis of covariancc on 
pretests and posttcsts, and correlations between student 
characteristics and programming outcomes, she found 
no significant difference in the programming outcome 
between groups and individuals. She concluded that 
group work did not harm the students, but it did not 
give beneficial help over individualization (Webb, 
1985a). 

Webb et al. (1986) conducted another study of small 
groups engaging in computer programming. This time 
she dealt with problem-solving strategies in learning to 
program in BASIC. She used 55 male and female 
students aged U-14 years and involved them in a ten- 
lesson workshop. This time, using frequency of 
behavior and partial correlations between group 
interaction and posttest scores, she found that group 
interaction was beneficial and that students learned and 
imitated each other's behavior. If the Webb studies were 
the on!y ones conducted on group use of computers in 
instruction, the inconsistency of results would have 
been a discouragement to further consideration of this 
topic. 

Research on Groups vs. Individuals 

Even without the mixed results of Webb's studies, some 
other studies have proven to be inconclusive. Most of 
the research cited deals with whether groups work better 
than do individuals, ^though some studies also address 
what type of group interactions are taking place. 
Karweit and Livingston (1969) tested the hypothesis 
that students who play a computer game in groups of 
two or three would per brm at least as well as students 
playing the game alone. They also surmised that the 
groups of students would be as successful as students 
working alone. They tested 44 sixth graders with high 
academic ability in four randomly assigned, sex 
segregated groups. One group played alone, one group 
played in pairs, and one group played in threes. The 
control group did not play the game. The Surfboard 
Game, developed by Jimmer M. Leonard of the Board of 
Co-operative Educational Services (BOCES), had 
students take on the role of a manager of a company. 
The students had to make managerial decisions about 
employees and company costs. A test was given after 
the game to see what was learned. 

A two-way analysis of variance was performed on Uie 
test* and the results showed no significant difference on 
learning in any of the group types. Other factors, such 
as using differen. inachines, also showed no difference 
in performance. The only difference cited was that boys 
tended to play the game faster than girls. 



Instructional Groupware 

Other studies, however, have given more positive 
results. Trowbridge and Dumin (1984) reported positive 
results when they conducted a study to see if groups 
learn better than individuals in a computer environment 
and whether this learning could be applied to non- 
computer tasks. Individual interactivity, as a function of 
group size, was examined by looking at various modes 
of interaction that students could use while working on 
the computer. Fifty-eight seventh and eighth graders of 
middle ability were placed in groups of one, two, three, 
or four students. The unnamed computer program had 
students conduct high-level learning activities by 
manipulating batteries, bulbs, and wires to perform 
experiments with electricity. Students took a pretest and 
posttest on their knowledge of electrical circuits. A 
paper-and-pencil test and an interview were given three 
months after completion of the program. In addition, 
students were to apply concepts to non-computer tasks 
by completing a circuit that lit a flashlight 

An analysis of variance and a mean score for videot24)ed 
and audiotaped interactivity across the four types of 
groups were compiled. Results showed that groups of 
two to three students work better at the computer than 
do individuals. Also, these students seem to interpret 
questions better than do individuals. Groups of four 
students seem to be too large because one or more 
members is left-out 

Another study was conducted by Hythecker et al. (1985) 
to see if students learned better cooperatively at 
computers than without computers. The authors 
developed and evaluated a training module derived from 
computer-assisted instruction and computer assisted 
cooperative learning (CACL) methodology. This 
module was used by 89 students at Texas Christian 
University who were randomly assigned to a CACL 
group, an individual learning strategy group, or a no- 
treatment group. The module was devised to teach 
students how to develop strategies for using 
MURDERl and MURDER2 by Danscreau et al. (cited 
in Hythecker et al., 1985). The training module was 
used for three, two-hour sessions. The first session was 
used to assign groups and start the module. The second 
session was spent working through the training 
strategics. The third session was used for testing. The 
CACL group used the module as it was designed, while 
the individual learning strategy group was given 
transcripts of the module and (old to work individually, 
and the no-treatment group was told to study the 
transcripts using whichever study method they prefeired. 

A two-way analyses of covariance between the Delu 
Reading Vocabulary Test (for verbal aptitude) and the 
Group Embedded Figures Test (for text processing 
performance}, was computed. In addition, t-tests were 
computed on lecall of total ideas, main ideas, and detail 
ideas. Hythecker et al. (1985) found the CACL group to 
have the best performance over the individualized or on- 
treatment groups. Subjects in the CACL group 
concluded that the learning strategies were more 
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beneficial and*ihat ihcy had more personal gain than did 
subjects in the individualized learning group. Tlie 
auihors stated that this study confirmed that cooperative 
learning and computer-assisted instruction were 
compatible learning methods (Hythccker et al.. 1985). 

Another aspect of group study was studied by 
Cart Wright and Cohen (1977). They investigated the 
learning and auitudcs that students had when working in 
groups with computers. They used a computer program 
written in CAN IV that was designed for three sessions 
of group work in introductory psychology over a one 
week period Twenty-eight males and females (education 
graduate students at McGill University) were randomly 
assigned to groups of three in three sections. The first 
section entered all work at the computer individually. 
The second section discussed answers and entered a 
consensus answer for the group. The third section 
discussed answers without reaching a consensus and 
entered their own answers. An attitude test was given as 
well as an unannounced criterion test. 

Using an analysis of variance on the criterion test 
showed no significant difference in mean individualized 
learning scores. However, an analyses of the mean 
number (by a Scheffc multiple comparison) of correct, 
incorrect, and unexpected responses did show a 
significant difference among the three types of groups. 
The group that gave consensus answers had more correct 
responses than the other two groups. Cartwright and 
Cohen (1977) concluded that oven though individuals 
working alone took less time than a group, the total 
groups within a class took less Ume to finish a task. 
Attitudes toward computer instruction were unaffected 
by this type of learning. 

Bellows (1987) conducted a study with younger children 
using computers and examined task performance and the 
social interaction of these youth. The ability and sex of 
66 second grade students were studied in relation to their 
achievement on a social studies mapping skill. Ability 
was designated as reading ability and class ratings by the 
teacher. In addition, this study also looked at student 
interactions amon{$ the group members. 

Three different classrooms were studied containing a 
total of 28 boys and 27 girls. Students were randomly 
assigned to groups of two or three. Students were to 
complete an unnamed computer program on social 
studies mapping skills and then play an unnamed 
computer game to determine if they had learned the 
skills. A pencil -and-paper pretest and posttest were 
given to the students in addition to the computer 
programs. Cronbach*s Alpha was used to determine 
internal consistency of the pretest and posttest. An item 
analysis showed adequate discrimination among test 
items. Reliability of the pretest and posttest was 
determined by administering it to 36 second-grade 
students not involved in the study. Verbal interaction 
was audiotaped and two observers recorded non-verbal 
behavior (Bellows, 1987). 



Results showed students did learn mapping skills from 
the computer programs. Sex appeared to have no affect 
on the achievement of the group members as a result of 
calculating an analysis of cov^ariance between the pretest 
and posttest. Students in groups of three had greater 
achievement gains in mixed-ability groups than uniform 
ability groups. Non unexpectedly, ability and 
achievement were significantly correlated. In nine of the 
22 groups, one student was dominant, and this student 
had the highest ability level. Also, five out of six times 
the dominant student was a male. At times, a group 
member was excluded from participation by the group. 
Usually, this was the low-ability student Students did 
not usually give or receive explanations, probably 
because of their limited vocabulary, but they spent a 
great deal of time with group members reading and 
copying from the screen. Bellows (1987) concluded that 
ability is a factor in group interaction and achievement, 
at least at 2nd giiide level. 

About the same time as the Bellow*s (1987) study, 
Allen (1988) became concerned that research studies 
were overlooking the social interaction of students when 
they worked on computers in small groups. She 
conducted a study with groups of students in grades four 
through eight to see exactly what social interactions 
took place. She started with 39 students in a pilot study 
that familiarized the students with an unnamed word 
processing software package and LOCK). A sample of 
18 high-ability, male and female students in groups 
already formed from the pilot stoidy were tested. 

The study consisted of three parts: (a) the observation, 
(b) a questionnaire, and (c) a structured interview. The 
observation was conducted over a two-month period 
with each grade level videotaped and audiotaped as they 
worked at the computers. A total of 10 hours (five 
separate session for each grade) was recorded. Next, two 
trained observers viewed and coded the social interaction 
tapes and characteristics for each of the groups 
determined during the pilot study as well as the amount 
of '^hands-on" time each student had. The second part, a 
questionnaire rated by a Likert scale and based on group 
behaviors and social interactions from the study, was 
given to the 18 students. Any unrealistic ratings by 
students were selected for further study. The third part, a 
videotaped and audiotaped suucuircd interview, consisted 
of a series of questions to determine why students 
responded the way tliey did on the questionnaire (Allen, 
1988). 

The results showed a great many social interactions 
(4,127) during the 10 hours of taping. Collaboration 
was the most frequent interaction, and collaboration 
increased as the grade level increased. Social talking 
did occur, but it was often in relation to the task and 
made up only a small portion of the interactions. A 
great deal of teacher interactions was included, but this 
lessened as the grade level rose. There was almost no 
antagonism, but there were a few incidents of isolation 
when the students became bored and psychologically 
withdrew from the group. Allen (1988) concluded that 
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positive social interactions resulted when the students 
worked in groups with computers and teachers also 
played an important role. Therefore* using computers in 
groups in classrooms might be beneflcial to both 
students and teachers. 

Yueh and Alessi (1988) believed that students tended to 
spend more time on group work than in individual work 
when working with computers. Because of this, they 
decided to see if a reward structure and ability 
composition would enhance attitudes toward group 
computer work, pecifically, Yueh and Alessi (1988) 
tested to see if groups receiving both individual and 
group rewards worked better than groups receiving only 
one type of reward. They also tried to determine if 
medium-ability students learned more in groups of 
medium-ability than in groups of mixed-ability. 

Seventy junior-high school students from three algebra 
classes in eastern Iowa were tested. The treatment was 
randomly assigned because the students could not be. 
Students took an achievement test with a reliability 
rating of .86 to determine ability. Class I was rewarded 
for individual performance. Glass G for group 
performance, and Class I-G for both types of 
performance. Within the classes, groups were formed to 
insure average ability of all group members. Males and 
females were equally distributed. Twenty groups were 
formed with an equal number of average-ability and 
mixed-ability groups in each class (Yueh & Alessi, 
1988). 

Each group used "Algebra Drill & Practice I for the 
Apple ir by Detmer and Smullen. The groups were 
audio-taped for coding analysis later. Each group was to 
practice using the program and then later a tournament 
would be held. Students could help each other and took 
turns answering questions. An unnamed computer game 
by Yueh was used for the tournament At the end of the 
tournament, students received prizes. The group was the 
unit of analysis. Results showed that Class I-G had 
higher achievement and more peer tutoring. 
Combination of individual and grcjp rewards, then, 
appeared to work best. Ability grouping was 
inconclusive (Yueh & Alessi, 1988). 

Newman et al. (1989) found that using computers in 
groups helped students and teachers enhance the learning 
process. They devised a prototype of a sixth grade earth 
science curriculum that used local area network (LAN) 
technology. The basis for the curriculum prototype was 
established practices of cooperative learning groups and 
inquiry activities. The LAN connected 25 Apple lie 
computers to a hard drive which allowed for central 
storage of group data. The Bank Street Writer word 
processing software had an electronic mail system added 
to it and was accessed by the LAN as was The Bank 
Street Filer. These software packages allowed students 
and teachers to create databases for access by all 
participants. 



Groups were selected by students and as a result became 
groups of the same sex. Some teachers were in favor of 
the study, some were reticent The group study lasted 
for one year. Students were observed in their group 
work and in whole-class woilc. The data that students 
entered as a group were combined with other groups* 
data and discussed as a whole (these data-gathering 
activities were called coordinated investigations). 

No statistical measures were taken, but quite a few 
observations were made. Newman et al. (1989) found 
that students worked well together. In fa^.c, the groups 
became quite cohesive. There were some problems with 
not all students pulling equal weight in the group, and 
there were times when sttidents wanted to work alone. 
However, these instances were not the norm. The 
teachers that had been reticent, as well as other 
cooperative teachers, started using more small-group 
activities in other subject areas. The groi^ were carried 
over to other subject areas and in other classes. 
Eventually, the whole school was affected by these 
small groups. Newman et al. (1989) concluded that 
cooperative groups were important to the educational 
process, but they felt individualization could be 
beneficial. Relevant to this discussion, the authors 
concluded that software designers and developers needed 
to design software that played a m(He supportive role in 
the small group instructional process. 

Problems to Overcome in Research on 
Using Computers with Small Groups 

Results from the studies cited often appear 
contradictory. For example, Karweit and Livingston 
(1969) found no difference in group size while 
Trowbridge and Dumin (19S4) found group size made a 
significant difference. Allen (1988) and Yueh and Alessi 
(1988) found groups to provide learning impetus for 
group members while Newman et aL (1989) also found 
groups to be beneficial and, ultimately, affecting the 
whole school. However, one similarity that all the 
studies had in common was that the group was a 
functioning part of the computer environment and 
groups caused social interaction among students. The 
groups may not always have been cooperative as the 
ones suidied by Webb (1984a, 198Sa), Webb et al. 
(196«V and Newman et al. (1989). In fact, they may 
even have been competitive as shown in the study by 
Yueh and Alessi (1988). In any event, the research cited 
above does indicate that groups are an iiaportant 
component in computer work. 

Studies need to be conducted within the context of 
instruction which can reasonably be expected to be 
productive using cooperative learning strategies. This 
means that an appropriate choice of teaching mode](s) 
would be drawn from those which have characteristics 
similar to Group Investigation (Joyce & Weil, 1986; 
Thelan, 1%7). If computer programming is the content 
domain of the insuiiction, it would be desirable to use 
small group process more in the same way that a design 
group tackles the task of solving a programming 
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problem. Very few major programming efforts arc one- 
person tasks; woric groups design, prototype, determine 
common parairiCtcrs for proceeding and divide 
responsibilities for development. This process more 
closely resembles the type of classroom activity most 
amenable to coq)erative learning practices. 

Furthermore, with a few exceptions (Jemstedt. 1983; 
Newman. 1989). the software used in the studies 
reviewed was designed for use by one student interacting 
with the computer as a substitute for the teacher, drill 
master, or a tutor (Yueh & Alessi. 1988). Many of 
the inconsistent findings might very well have been 
eliminated by the development of software better suited 
to use by small groups. 

Con)put3rs and Small Groups 
In Business and Industry 

There are those who would argue that at least to some 
extent school learning environments should minor the 
work contexts of the "real world." While this is not the 
premise upon which most group use of computers in 
instruction is based nor is it the premise underlying 
cooperative learning models of leaching as with Group 
Investigation (Thelan. 1967; Joyce & Weil. 1986). it 
should be noted that few jobs in today's work world are 
solitary activities. Groups are a basic part of society. A 
new trend that is emerging in business and industry is 
the active collaboration of groups and group members 
through the use of computers and group-oriented 
software. This group-oriented software is labelled 
"groupware" (Tazclaar. 1988; Terry. 1989). It often 
involves neiworicing between different locations and is 
used with groups or with individuals sharing 
infonnaiion. For example, a document is created and 
then, via the network, is passed to several people for 
additions and editing. The concept is not new. but its 
implementation enhances cooperative group woiic in the 
business and industry. This is the same type of 
computer usage that Newman et al. (1989) studied in 
their earth-science curriculum. 

It would seem that more woric with groups in schools is 
needed to support a practice that is already common in 
the business world arid to reinforce or refute the research 
already done on groups in schools. As Johnson. 
Johnson. Holubee. and Roy (1984) succinctly stated. 
'We cannot afford to graduate large numbers of students 
with little or no ability to interact effectively with 
othcrs-a prime requisite in the world of work" (p. 11). 
The need is also present to investigate the cooperation 
of groups woricing at computers, because computers as 
well as groups are inevitable parts of the "real worid." 

If we are willing to accept the assertion that cooperative 
learning should be encoi'.'aged in today's classrooms for 
at least some types of curricular goals, and if we take 
seriously the potential of facilitating a very human, 
face-to-face interactive process with computers, then it 
becomes essential to look at principles which may 
undergird the development of software appropriate to the 
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small group environment. It is interesting to note that 
the successes reported in some studies on group use of 
computers have been achieved using software eloped 
specifically for single user interfaces wiucn often 
attem^ t to emulate the dialog typical of a one-to one 
interaction of a student with a teachers. It is only 
logical that software designed specifically for small 
group use should engender more success. The principles 
needed to guide design of software for cooperative 
learning environments must be derived from a generic 
set of criteria for successful cooperative learning and 
from criteria for teacher roles which enhance successful 
cooperative learning. 

Criteria for Successful 
Cooperative Learning 

According to Johnson and Johnson (1984). the most 
effective of group processes (and the least used) is 
cooperative learning where students work together u> 
complete shared goals. In order for cooperative learning 
to be successful. Johnson and Johnson (1984) feel four 
criteria must be met 

1 . Positive interdependence must be achieved by 
group members. Each member must feel good 
about the role played within the group. 
Elements of declaring interdependence within 
the group are 

• Establishing mutual goals (goal 
inierd^iendence); 

• Dividing labor (task interdq)endence); 

• Dividing materials, resources, or 
information among group members 
(resource interdq)endence); ' 

• Assigning students roles (role 
inteniq)endence); and 

Giving joint rewards (reward 
inteFdq)endence). (p. 1) 

2. Group members must have face-to-face 
interaction. The verbal and non-verbal 
interactions among group members are equally 
as important as establishing positive 
interdependence. 

3. Each group member must accept individual 
accountability for completing abortion of the 
group's work. After all, cod^ttatiVe learning in 
schools is used to help each student learn. The 
group must decide the limitations of each 
group member and then determine realistically 
what each member is capable of handing. 

4. Group members must use interpersonal and 
small group skills appropriately. The teacher's 
role in relation to the group is to teach 
members the limitations and boundaries of the 
group, acceptable social skills, goal setting, 
and the skill to analyze progress of the group. 
Cooperation as opposed to completion is an 
important group norm which must be learned 
by participants in the process. 
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The Teacher's Role for Successful 
Cooperative Learning 

Just as imputtant as the group members establishing 
their roles and goals, is the role that teachers play 
throughout the cooperative learning process. Johnson 
and Johnson (1984) caution teachers that they need to be 
aware of what is happening in cooperative groups and 
make sure the learning process is enhanced and not 
harmed. These authors have set up five teacher strategies 
to insure beneficial cooperative learning. A sixth has 
been added by the author of this p£^r. 

1. Clearly identify the lesson's objectives. This 
means not only academic objectives 
appropriate for the lesson but also objectives 
incorporating group skills, roup members 
should clearly understand what is expected of 
them. 

2. Determine how groups will be constructed. 
Several factors should be considered in group 
construction. 

• Group Size: Should groups have two 
members or up to six? What size is 
appropriate for the lesson, the time and 
materials available, and the experience of 
members working in groups? Johnson and 
Johnson (1934) suggested that unless 
group members are experienced in 
cooperative group work, group size should 
be no larger than two or three members. 

• Group ability: Should homogeneous 
ability groups or mixed ability groups be 
used? 

• Time group members will work together: 
Is the lesson ^propriate for long term 
groups or should new groups be made 
everyday. 

• Room arrangement. Will group members 
be able to face each other and talk quietly? 
Will the teacher have easy access to each 
group? 

• Instructional materials: Should each 
member have instructional materials or is 
one copy per group sufficient? Johnson 
and Johnson (1984) felt that one copy per 
group is a better way to promote 
cooperation and collaboration. This way 
each member receives only the portion 
necessary to do the job. 

• Group member roles: Does each group 
have a member who summarizes the 
goals, a member who checks to make sure 
all other members can answer questions 
about the goals, a member who checks for 
mistakes, and a member who can insure 
that group members can relate the new 
goals to previously learned goals? 

3. Expf un the task and cooperative goal structure. 
This strategy requires effective communication 
on the part of the teacher. The teacher must 
explain the academic task, the teacher should 
make sure the group members understand that a 



group goal must be reached, that they must 
work collaboratively and with certain desired 
behaviors, and that each group member is 
accountable for a certain portion of the group's 
goal. The teacher should explain whether one 
product must be produced or whether each 
individual should produce a product and what 
evaluation criteria will be used. 

4. Momtor group effectiveness and intervene to 
provide academic and/or group process 
assistance. This is a time when groups should 
perform the tasks necessary to reach the desired 
goal. Teachers should monitor carefully, 
though, to make sure that groups are working 
cooperatively and that there is some sense of 
closure when the lesson is completed. 

5. Evaluate group achievement and assess how 
well members collaborated together. The 
quality and quantity of the group product 
should be evaluated. In addition, group 
members need to understand if their 
collaboration was productive. This productive 
collaboration is most important for future 
group work. 

6. Emphasize concensus as essential to group 
success. In order to produce a worthwhile 
product, members have to reach some 
concensus. Concensus is often difficult to 
reach, but group members learn to support all 
positions taken and to generate new solutions 
to problems which may well be different than 
the solution of any individual memb^. 

Guidelines for Design of 
Instructional Groupware 

What do these principles mean for the design of 
instructional software, or« to borrow a term from the 
non-educational software community, the design of 
instructional groupware? In order to deal with this 
question it is important to consider the apparent 
problems with cunent software design when the 
resultant software is considered for use in the small 
group context In most of the studies cited above, vague 
references were given to the software. In fact, the 
software was not even named in four instances, except 
its type such as drill and practice. The software may 
play a big role in determining the results of group 
research with computers. 

If the assertion is made that software is important, then 
who decides what software is good or bad? Fisher (1982) 
discovered through interviews with research scientists, 
computer and software manufacturers, educational 
publisners, and school administrators that the largest 
group of decision makers about software were the 
computer and software manufacturers and educational 
publishers. Disturbingly, he also found that educational 
publishers felt it was their role to provide computer 
software to educators, and quite often the software 
provided was nothing more than a textbook on line. In 
contrast, computer and software manufacturers produced 
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software, but they frequently were not aware of sound 
teaching principles. These publishers felt the schools 
should know enough about computer-assisted 
instruction to decide whether the software was good or 
bad (Fisher, 1982). 

Fortunately, some of the bad software has disappeared 
("The 1988 Classroom," 1988) since Fishers study 
(1982). However, the evaluative criteria for identifying 
good software may still not be sufficiently precise. 
Becker's (1984) survey stated that work done at 
computers in schools was done by groups of students 
rather than individuals. However, in two magazines 
which summarized the best software of 1988, both 
overwhelmingly described software that was clearly 
designed for individual use ("User-friendly," 1988; "The 
1988 Classroom," 1988). In "User-friendly Pages" 
(1988), only two software packages were identified as 
appropriate for group use out of the 500 described. In 
"The 1988 Classroom Computer Learning Software 
Awards" (1988), three of 46 software pages were 
described as appropriate for group use. Clearly, there is 
a major difference between the audience for whom the 
software is being designed and how the software is in 
fact being used in schools. 

In some textbooks that teach instructional software 
design and development strategies, the emphasis is 
placed only cn individual use. Hannafin and Peck 
(1988), staled in the beginning of their text that 
software should be developed for individualized use and 
their book was based on this premise. Heimler, 
Cunningham, and Nevard (i987), emphasized the actual 
writing of code to produce software. However, there was 
no mention of incorporating cooperative learning in any 
software examples .that the authors used. Ironically, 
both of these books were written by more than one 
author. At least two people collaborated to produce each 
textbook that is intended for individualized software 
design and development 

What is even more ironic is that most software is 
designed and developed by more than a single person. 
For example, Shneiderman (1987), a leader in the field 
of human-computer interaction, devotes a whole section 
in his book on assessment of software by groups. He 
firmly believes that iterative design is an integral part of 
effective software design. Iterative design, as he explains 
it, involves developing a prototype and then 
scrutinizing that prototype to make sure it meets the 
requirements of its intended audience. This process 
usually requires the participation of a number of people. 
One person is not responsible for the entire production 
of a software interface. Another example of group work 
to produce software has been done by Good, Spine, 
Whiteside, and George (1986). They describe the 
iterative design process used to develop a revised version 
of a software package used on a VAX machine. The 
team of authors developed the revisions, and then a 
group of users scrutinized the revised software. 
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Designers of instructional groupware should visualize 
the dynamics of small group process (Thelan, 1967) and 
the specific role the software under development will 
play, ontextually, small groups will engage in: 

1) collaborative communication (verbal & non- 
verbal). 

2) concensus building within group. Software 
designers must assume that a concensus 
building process will be a part of the group 
process and that individual responses must be 
negotiated prior to inputting responses U 
computer (or, if ICAI, that software will nol 
take over the concensus process since this is an 
essentia] part of two face to face encounter of 
small group woric 

3) group response to one or more discrepant 
events. It may be necessary for, the software to 
first present a discrq)ant event in such a way as 
to challenge the group to engage in problem 
solving behavior, data collection, hypothesis 
testing and the like. Group inquiry and 
problem solving are the norm for working 
groups. 

4) hypothesis testing. The software will need to 
provide feedback at multiple stages of the 
group process. This will be especially true in 
the testing of alternate hypotheses proposed by 
the group. Feedback may need to lay out 
probable consequences of given hypotheses 
rather than give the typically "right or wrong" 
response. 

5) a different kind of interaction. This may require 
extended use of parsing, much as in old 
"adventure* games to allow user groups to try 
discovered facts and relationships in the fcnm 
of positively worded assertions, followed by 
feedback with new stimuli 

6) extensive record keeping and the organization 
of reorganization of ideas. The computer is 
ideally suited to facilitate this task, but 
structure of specific softwa^ must provide this 
capability or provide an easy interface with 
tool software suited to this purpose. 

Instructional design principles such as those 
promulgated generally (Gagn6, Briggs & Wager, 1988; 
Dick & Carey, 1985) and those specifically pertaining 
to software design (Hannafin & Peck, 1988; Heimler, 
Cunningham, Nevard, 1987; Shneiderman, 1987) will 
still be essential for sound approaches to groupware 
development. However, several factors will confound 
strict adherence to these principles. Generally, the 
objectives for instruction conducted within the 
cooperative learning mode are more process oriented 
than specific-outcome related. Secondly, to set out to 
develop groupware assumes a perspective of curriculum 
which values the process of cooperative learning and the 
nurturant outcomes which are often more important 
than the anticipated direct outcomes (Joyce & Weil, 
1986). This means that the overall instructional strategy 
calls for group process, and the software will be used to 
facilitate this process. 

3Q0 



Furthermore, the design phases of formative and 
summative evaluation may need considerable 
reconceptualization. Given the perspective of groupwaie 
which facilitates a process in which face-to-face 
interaction occurs among group participants and 
accommodation to the unpredictability of learning 
outcomes (Thelan, 1960), outcome measurement seems 
to be less of a criterion for evaluation than is an 
observable index of group process variables. The extent 
of interaction, the quality of that interaction, the 
perceived commitment of group participants to 
concensus reached, the amount of live intervention by 
the teacher to assure fruitful continuation of the small 
group process are just some of the variables which must 
be considered as bases for formative and summative 
evaluation aspects of design. It will be necessary to 
assume that what individuals learn is primarily a result 
of the group process. To identify the specific 
contribution to such learning outcomes of the computer 
usage and of the specific software used may not be 
necessary. 

Already, a few group-oriented software packages exist 
For example, the CACL technique discussed by 
Jemstedt (1983) seems to be true cooperative learning 
software. Only an overview was given on how the 
CACL software actually operated, but if it did do the 
things Jemstedt (1983) professed, then it would be a 
beneHcial addition to a curriculum. Group members 
entered their responses to a question, and then the 
answers and the strategies used to arrive at particular 
answers were discussed collaboratively by the group. 
The software allowed time for collaboration before the 
next question ^>peared. In fact, the group had to jointly 
decide when it was appropriate to move to the next 
question. 

The earth science software package with LAN 
capabilities presented by Newman et al. (1989) also 
shows great promise for cooperative learning. Not only 
do groups of students record data into a shared database, 
but the data collected from the whole class can be 
manipulated and discussed by each small group. In 
addition, the whole class can perform a type of 
consensus by using the information gathered from each 
of the groups. The electronic mail portion of the 
software also promotes collaboration between groups, 
instructors, and group members and is very similar to 
the concept of "group ware" in the business industry 
presented by Tazelaar (1988). It sounds as if all aspects 
of cooperative learning, from small groups to whole 
class, is possible with this software package. 

Two oth» examples of group software already available 
are "Story Tree" by scholastic and "The Factory" by 
Sunburst. With "Story Tree" ("User^friendly," 1988). a 
team of students produces a story. The story is 
interactive and according to what the reader uses as 
input, determines what the next segment of the story 
will be. "The Factory" (Vockell & van Deusen, 1989) 
allows a product to be made from specifications given 
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by students. According to Vockell and van Deusen 
(1989), this software works well with groups because 
the group members can collaborate on what 
specifications are best to produce a specific product. 
Vockell and van Deusen (1989) also suggest using this 
software as a whole class group project. Small groups 
can come up with collaborated specifications and the 
whole class can reach a consensus about the best 
specifications to use to design a product 

More such examples of groupware are needed. Careful 
adherence to the principles of cooperative learning and 
to the practical implications for groupware design 
alluded to in this paper can provide a good starting point 
for such development. More importantly, the research 
on groups using computers: I) must be redirected to be 
conducted within the context of curricular situations 
which dictate small group process; and 2) must use 
software which has been designed specifically to 
facilitate cooperative learning. Only then can we hope 
to see consistency of results from this needed field of 
research related to instructional design. 
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THE EFFECTIVENESS OF THE CAI DRILL DESIGNING ON STUDENTS WITH 

DIFFERENT LEARNING STYLES 



Jon-Chao Hong and Nan-Shin Yen, National Taiwan Normal University 



Abstract 

A guiding principle in teaching has been the modelling 
of humanitarian values through individualized learning 
environments in which the needs of every learner are 
addressed Now, as the methods of artificial intelligence 
are applied to computer-assisted instruction, the the 
seemingly improbable dream of truly individualized 
instruction is becoming a reality through intelligent 
tutoring systems. The purpose of this study was to 
examine the effects of different modes of CAI drills and 
their impacts on learners with varying learning styles. 

The sample of this study was consisted of 1303 K7 and 
K8 students. After initial selection, 40 English 
vocabulary lessons (20 introductory and 20 advanced) 
were chosen as stimulus materials for uie comparison of 
drill times (Drill 2, Drill 3, anc Drill 4). After 
completing a learning styles invento y and pilot study, 
the original sample was reduced to 171 students. 
Dependent me^^-^ ures consisted of The final test on 
computers included: Literary and Scientific Aptitudes 
Test; Keyboard Speed Test; and Drill Practice. 

The study produced to the following conclusions: 1) 
Different durations of drill practice effected independent 
learners, but had little effect of dependent learners; and 
2) Different drill strategies are more effective than 
others, regardless of learning style. The findings of this 
study may be useful in CAI drill design. This study was 
sponsored by the Chinese National Science Council. 



Introduction 

The development of CAI courseware has traditionally 
relied primarily on the application of fundamental 
conceptions of educational psychology. However, 
traditional instructional design has often neglected to 
incorporate many of these principles within courseware. 
In other words, the effectiveness of CAI dq)ends not 
only on the developmwii of software and hardware, but 
also on the exploitation of data processing, human 
cognition, emotion, and skill. Therefore, it is 
imperative to study the variables effecting CAI 
courseware design in order to produce software that not 
only incorporates the principles of sound teaching 
practice, but also matches the student's aptitude and 
learning styles or preferences. 

Good (1987) observed that the development of ICAI has 
made the application of more complex and obscure 
pedagogical conceptions of cognitive science possible. 
The purpose of individualized ICAI is to provide 
learners with opportunities for self-improvment (Stabbs 
& Piddock, 1985). However, to fulfill this function, the 
courseware must be flexible and be able to adapt to 
satisfy students with different backgrounds and 
aptitudes, as well as provide an humane learning 
environment. A variety of scholars have suggested 
essential factors that should be considered in ICAI 
courseware design, including: knowledge bases, student 
models, pedagogical models, and user interfaces (Dede, 
1986); and pedagogical control, learner characteristics, 
pre-knowledge diagnosis, and learning approach 
(Jonassen, 1985). Rosenburg (1987) also emphasized 
the importance of the study of learners characteristics 
and learning styles. 
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Combining the theories of both Jonassen and 
Rosenburg, the indispensable factors in ICAI 
courseware designing could be classified into the 
categories shown in Table L 

In the student model, the study of the student's learning 
style, modelled closely after cognitive psychology 
principles, could enable ICAI courseware to meet the 
student's individual needs.As the preceding analysis 
revealed, the study of the student's learning style is a 
prerequisite in designing effective ICAI courseware. The 
objective of this study was to examine the interaction 
between different learning styles and different ICAI 
pedagogical models. 

Purpose 

The major guiding principle of modem humanitarian 
education is to regard the student as the central concem 
and guide the student through the process of learning. 
Therefore, the student's aptitude, interests, and learning 
style must be all taken into consideration. Similarly, in 
order to match the student's aptitude, interests, and 
learning style to the instruction in a CAI environment, 
it is necessary to examine the degree to which different 
CAI models account for learning style effects This study 
concentrated on exploring the various leaming responses 
and achievements of students with different learning 
style in several CAI drill models. 

Research Questions 

The first part of this study focused on the frequency of 
drill repetitions. Since the purpose of a drill is to 
increase leaming effectiveness through repeated practice 
based on a stimulus-response theory of behaviorism, the 
frequency of repetition should have a direct effect on 
achievement (Skinner, 1986). The study included the 
following research independent variables: the student's 
literary or scientific aptitude, the basic speed of 
operating keyboard, and leaming styles. Moderator 
variables included: the ofrequency of drill and the 
difficulty of the teaching materials. The dependent 
variables was a measure of leaming achievement 

Hypotheses 

According to the research questions above, the four null- 
hypotheses of this study were constructed as follows: 
} q: There is no significant difference in leaming 

among students with different leaming styles 

under different frequencies of drill practice. 
H02: There is no significant difference in leaming 

achievement among students with different 

leaming aptitudes. 
H03: There is no significant difference in response 

time among students with different leaming 

styles under different frequencies of times drill 

practice. 

H04: There is no significant difference in error rale 
among students with different leaming styles 
under different number frequencies of drill 
practice. 



Review of the Literature 

Many scholars have defined the concept of leaming style 
and proposed different criteria for classifying leaming 
styles. These criteria include: teaching style, classroom 
administration, development of individual cognition, 
development of individual emotion, individual 
temperament, etc. Wilkin (1976) observed that, during 
the process of leaming, the interaction between the 
learner and the leaming environment can have dramatic 
effects on leaming. Consequently, he proposed two 
overarching leaming styles: field-dependent and field- 
independent 

Wilkin's classification schema has been applied in many 
studies related to CAI. Smith's (1985) study of 
microcomputer-assisted instruction, for example, 
concluded that field-indej>endent learners were more 
compatible with the individualized teaching 
environment of CAI. This viewpoint was further 
emphasized in Post's (1985) study. Cosky (1980) 
suggested a relationship between cognitive style and 
CAI. Specifically, he noted that field dq)endence, even 
more that inteelligence, was the key factor to consid^ 
in matching leamer's with appropriiate lesson designs. 
In studying the effects of leaming style on CAI in 
nursing education, Ludwing (1986) pointed out that 
learning style in different CAI courses had a 
considerable influence on the leaming attitude of 
leamers of different ages. Moreover, using a three-tier 
classification scheme for leaming style (auditive, tactile, 
and visual) Martini (1986) concluded the tactile leaming 
style is most compatible with conventional CAI 
programs. He also emphasized that the compatibility 
between leaming style and teaching program might have 
a critical effect on the attitude and achievement. 
Massoumian-Zava (1986) studied the achievement of 
leamers' with different leaming styles und'^-r conditions 
of various types of feedback and concluded that field- 
dependent leamers are best suited to affective- 
informative feedback style. 

As the literature shows, leaming style has indeed been 
an important element in the efficacy of CAI courseware. 

Method 

Sample. The sample for this study was drawn from 
1303 1 ^ and kS students. They were administered a 
computer-based version of Wilkin's Embeded Figure 
Test, which included 32 figures. Students were classified 
as field-independent as field dependent. FrOTi the overall 
sample, 171 students were selected for further study. 

Materials. This study used the frequency of drill 
repetition to analyze the diversity in leaming style. 
Following the work of Dennis and Kansky (1984), the 
frequencies of drill repetition selected for this study were 
two (Drill 2), three (Drill 3), and four (Drill A) times. 
To avoid memory effects, new content material was 
selected for each of the drills as follows. 57 English 
'vords were selected from dictionaries. Words with 5 
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letters are regarded as "easy," ana those with 9 letters 
difficult vocabulary (Miller, 1967). The 57 words were 
piloted with 50 freshmen and sophomores familiar 
terms were eliminated from the list. The remainining 
words with their Chinese explanations were then piloted 
with 25 seventh and eighth-grade junior high students, 
resulting in the elimination of 10 additional terms. 
Theiefore, the final lesson content was composed of 40 
terms and definitions, 20 easy and 20 diflTicult. 

Dependent Measures, This study used Witkin's 
Embeded Figure Test of field-dependence/field- 
independcnce to evaluate the result of the experiment. 
This study used descriptive and comparative methods to 
address the following three issues: 

The test of learning style: 1303 examinees, including 
668 males and 635 females were tested by the Embeded 
Figure Test of learning style. The results showed that 
the average score of males was 5.53 (SD=5.96), the 
average score of females was 4.83 (5Z)=5.53). 

Length of the lesson segments: According to the result 
of the Endedded Figure Test, 120 students whoso 
average score was between 3 and 7 points were divided 
into 6 groups to complete the drill lesson. Each group's 
answers to the questions were recorded during a 20 
minute interval, and the result of this pilot study 
indicated that the number of questions should be set at 
11 for Drill 2. 9 for Drill 3,7 for Drill 4. 

The final tests: After the number of questions was 
determined, 171 students were tested by the following 
three kinds of final tests: a) Literary and Scientific 
Aptitudes Test; b) Speed Test of Operating Keyboard; c) 
Drill Testing. 

Findings 

Tested by various statistical methods, such asData from 
this study were analyzed through One-Way/Two-Way- 
ANOVAs, and t-tests. The study led to the following 
results: 

There was a significant difference in learning 
achievement field independent learners under different 
frequencies of drill practice (F=6.396, p<.05). 
Specifically, field independents performed better on 
shorter drills (Drills 3 and 4) that longer drills (Drill 2). 

Students with a literary aptitude and those with a 
scientific one showed no significant difference in 
learning achievement (r=1.48, p>.05) and different 
levels of scientific aptitude had no significant influence 
on the students' achievement (r= -1.71,p>.05).3), nor 
did different levels of literary aptitude (r= 0.86, p>. 05). 

Under different frequencies of drill repetition, students 
with different learning styles showed no significant 
difference in response time. Variations in drill frequency 
also failed to produce significant response time effects. 



Results of this study also indicate that students with 
different learning styles show a considerable difference 
in error rate (F=8.744, /?>.05), with filed independent 
Icamers outperforming field dependent learners. 
Moreover, the error rates tend to increase with the 
length of the drill. 

Conclusion 

The results of this study suggest the following points 
for future design of CAI drill-based lessons: Moderate 
length drill exercises should be used with field- 
independent learning styles to attain a better learning 
effectiveness. Longer drills may not only be ineffective, 
but may cause a decrease in learning due to boredom or 
other factOTS. 

The frequency of drill repetion seems to have little effect 
on students with a field-dependent learning styles. 
Therefore, in designing CAI drill for field-dependent 
students, for the sake of lesson efficiency, designers 
should also limit the length of drill lessons to their 
minimum effective length. 

Overall, the results of this study suggest that drill-based 
lesssons can be an effective means of teaching learners 
of varying learning style. However, these results also 
indicate that, once ooverleaming has occurred, the drill 
should be terminated to avoid a decrease in the overall 
lesson efficacy. 

These results again highlight the tremendous 
pedagogical importance of adaptive, intelligent tutoring 
systems that can effectively learn from learners and 
modmte the lesson according to learner characteristics. 
These systems hold much promise for promoting the 
dream of truly individualized, humane learning systems. 
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CREATIVE APPROACHES TO KNOWLEDGE TRANSFER 



Cynthia B. Leshin, Southwest Center for Instructional Development 



The purpose of training is to improve on-the-job 
performance. Most trainers assume that their students 
will be able to transfer the skills and knowk i learned 
in the classroom to the workplace. However, there is 
considerable evidence that less than ten percent of 
training expenditures pay off in actual behavior changes 
on the job. Much of what is learned in the training 
environment can only be applied to problems or events 
similar to those encountered in training. Equally 
disturbing is the fact that many training environments 
do provide the skills and opportunities necessary to 
facilitate transfer, however, the application of these 
skills is inhibited by the workplace environment. A 
familiar type of transfer failure would be the manager 
who masters organizational development skills in the 
classroom but fails to apply them successfully in the 
workplace. 

A new upsurge in the interest of transfer of training has 
begun to occur as most organizations tighten budgets 
and look for ways to reduce organizational spending. 
Business and industry can no longer afford to invest 
billions of dollars a year on training only to be 
disappointed in the resulting on-the-job performance. 

Transfer of training principles are urgently needed for 
training systems that teach complex jobs. The level of 
complexity for many training tasks requires that 
training be done in a simulated environment that 
provides practical experience in the development of 
efficient and safe skills. The uaining of military 
personnel, police officers, and physicians are but a few 
examples of training where it is less likely that training 
can be successfully done on-the-job. Consequently what 
is needed is a better understanding of the issues involved 
in transfer of learning. 

This paper discusses who is responsible for skill 
performance and transfer, how transfer of training 
strategies differ from traditional training methods, 
strategies for the transfer of training, and how 
technology can be used lo enhance transfer. 

Who is Responsible for Skiii Performance 
and Transfer? 

There are many people involved with facilitating the 
transfer of training to the work environment. The most 
critical people, however, are the trainer, the trainee, and 
the manager/supervisor. The trainer must consider the 
role of the manager in facilitating behavioral changes. 
Strategies for transfer must therefore begin prior to the 
training session and must continue after training has 
been complete. The role of the manager can not be 
overemphasized in promoting the application of the 
newly learned knowledge and skills. 



How do Transfer of Training Strategies 
Differ from Traditional Training Methods? 

Traditional insuiictional methods assume that learning 
is more or less under the control of the instructional 
treatment Such methods typically focus on the use of 
performance objectives that guide the design of the 
instruction, practice activities, and test questions. The 
problem does not lie in the use of criterion-based 
instructional methods, but rather in the way in which 
the objectives are written. Ineffective training frequently 
results from the selection and development of 
performance objectives that do not take transfer and the 
direct application of knowledge into account. For 
example, an objective for an insuiictional design course 
might be, "The learner will be able to state seven 
characteristics of the adult learner." To better represent 
the application and transfer of this knowledge the 
objective should read, "The learner will design a training 
program that incorporates the five principles of adult 
learning." The difference between the two objectives is 
that the second requires the learner to use a higher-level 
skill in the application of the knowledge rather than just 
the rote recall of information. 

Performance objectives that involve logical, systematic, 
and linear knowledge, such as procedural knowledge, are 
best suited to behavioral training methods where 
learning is achieved through rote practice. However, 
training that involves the application of concepts and 
principles to previously unencountered situations 
requires a more creative, intuitive approach where 
higher-level thinking skills are stimulated. Most 
performance objectives and their subsequent assessment 
do not take into consideration the development and use 
of these higher-level skills and their importance related 
to the problem solving and transfer of learning. 

Recent research has advocated the use of cognitive 
models that place more direct responsibility for learning 
in the hands of the student. Cognitive instructional 
methods take into account individual learning needs and 
continually adjust to accommodate different learning 
styles. These methods include the use of accelerated 
learning techniques, discovery strategies, cooperative 
learning, and techniques to enhance the development of 
self-regulatory, metacognitive skills. 

Cognitive strategies encourage students to use 
previously acquired knowledge and skills to assist with 
the integration and application of new knowledge. Such 
models advocate the develcpiT.ent of performance 
objectives that provide for the application of newly 
acquired skills and knowledge through creativity, 
intuition, discovery, and participative learning 
experiences. An instructional event under .the cognitive 
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model would* advocate practice activities where the 
learner applies the new knowledge to a variety of tasks 
and where feedback on the correctness of thinking 
becomes a critical component of the learning process. 

Strategies for the Transfer of Training 

There are three classifications of s!rategies for 
in proving the transfer of knowledge: pre-training 
straegics, training strategies, and post-training 
siiatigics. 

Fre-Training Strategies. Generally the need for 
training is determined in a needs analysis or a 
performance analysis. This preliminary process provides 
the greatest resource for support in helping to ensure 
that training will be transferred back to the work 
environment This front-end analysis identifies not only 
performance deficiencies but should also determine 
performance standards, and identify future needs. 

In instructional system design models, the needs 
analysis is the first step used to identify the performance 
problems that training is to address. Managers are in the 
best positions to help identify what these deficiencies 
are. Throughout the needs analysis process, strategies 
should be used to encourage management to buy into 
the training process. At the same time managers should 
be encouraged to participate in training to help impact 
employee performance. One methods is to have 
managers identify problem areas in their work 
environment. From these problems, case studies can be 
developed that can be used in training as transfer 
activities to support the analysis and synthesis of 
content. 

Managers are also in the position to help provide the 
needed information on performance standards and the 
identification of future needs. Needs might include 
expanded work responsibilities, problem solving, and 
changes in company operation. Each of thc<;e areas are 
critically important to the success of a training 
program. 

Throughout the analysis process there must be 
continued emphasis on how training can benefit 
performance, productivity, and profitability. In terms of 
transfer, nothing is more important than a manager who 
is supportive and excited about training his employees, 
and who personally takes responsibility for seeing that 
his employees are able to implement the newly acquired 
knowledge and skills. Managers can and should be 
included in every step of training design, development, 
delivery, evaluation, and reinforcement 

Training Strategies. Most strategies for improving 
the effectiveness of training are implemented during the 
traip'ng program. The following are generally accepted 
as being effective when used during training: 

Providing trainees with performance 

objectives. 

• Involving the trainee in the learning process. 



• Providing opportunities for practice and 
feedback. 

The use of questions. 
The use of case-studies. 

• The use of different media. 

A paradigm shift is beginning to occur as educators and 
trainers begin to explore new alternatives to old 
traditional learning methods. Interest in accelerated 
learning (AL) is growing throughout the world as 
companies become interested in learning systems that 
claim:. 

35 '50 percent increase in learning 

40-66 percent reduction in training time 

Less fatigue for students and instructors at the 

end of a long day compared to traditional 

training 

• Increase in: 

- motivation to take additional training 

- self-confidence in subject matter 

- overall self-esteem 

- ability to use resources. 

There are many different elements to accelerated 
learning, however, the main shift is from a learning 
environment that is instructor centered to one that is 
learner centered. No one element can defme accelerated 
learning. Techniques blend together into a learning 
environment where there is participation, cooperation, 
and diversity in learning experiences. Accelerated 
learning techniques can be classified into three groups: 
mental, psychological, and physical. 

• MENTAL. We know that there can be no learning 
without memory. However, frequently our learning 
is left behind in the classroom when we fail to 
effectively encode new knowledge. Accelerated 
learning techniques include the use of memory aids 
such as visual associations, imagery, and mind 
maps to help organize material and improve 
retention. Techniques are used that stimulate both 
sides of the brain to encourage whole-brain 
learning. Knowledge bridges are created between 
new knowledge and prior knowledge. 

• PSYCHOLOGICAL. Educational psychologists 
have over the last few years been more interested in 
applying theories of how the brain really works to 
the learning process. Conventional education has 
assumed that learning would occur with 
concentration, practice, and feedback. We now are 
beginning to recognize that each individup/i has a 
different learning style and that a variety of learning 
techniques must be employed to address individual 
differences. 

• PHYSICAL. To achieve optimal learning 
outcomes the learning environment in the 
accelerated classroom is u^ansformed. DisU'actions 
are eliminated, including pictures or images on the 
walls that are not relevant to the subject being 
taught. Room decorations are replaced by colorful 
posters summarizing key points. Subliminal 
messages on many posters support learning as a 
positive, enjoyable, and challenging experience. A 



ERIC 



358 



388 



very important aspect of this optimal learning 
environment is providing a break every 10 mmutcs 
combined with some physical exercise such as 
stretching. At breaks, nutritious snacks are served; 
icilned sugar and caffeine is avoided. 

Accelerated learning offers a new and refreshing 
alternative to traditional instructional methods at a 
critical time when business can no longer afford to 
waste dollars on ineffective and inefficient training 
methods. 

Post-Training Strategies. Training follow-up is a 
critical component of knowledge transfer. The manager 
once again is the key resource to successful 
implementation of new knowledge and skills. If 
managers arc to be primary reinforcers of tiairiing, then 
they must be informed about the objectives and expected 
outcome of the training program. Ideally, this process 
begins in the needs analysis, however, it may need to 
again be rt-emphasized. A well informed manager who 
supports training will be more likely to offer 
encouragement to employees to use and implement new 
practices. The trainer may have to spend time with the 
manager to discuss how to help reinforce training. One 
method would be to offer a special workshop for the 
managers where objectives, outcomes, and 
implementation issues are discussed. 

Another important post-training strategy involves 
discussion and feedback on implementation practices. 
One method is to have a three or four week follow-up 
program where each employee gives a shoit presentation 
on his use of the training content Discussion may 
focus on how it has helped performance and on 
problems that may have been encountered. This process 
provides accountability for the transfer of knowledge and 
skills to the woricplace and is also an effective strategy 
for post-training review and assessment of training 
effectiveness. 

How can Technology Assist with 
Transfer? 

Learning is a process of discovery. Training that 
provides opportunities for exploration, experimentation, 
and the cooperative sharing of ideas facilitates the 
development of higher-level thinking skills. Such 
activities encourage the application and understanding of 
information through analysis, synthesis, and evaluation. 

One technology that enhances the development of 
higher-level skills is computer-based simulations. 
Students solve problems, learn procedures, observe the 
results of their chosen actions, and develop an 
understanding of complex phenomena in a safe and 
efficient learning enviro«^ment. What contributes to the 
power of a simulation to enhance transfer is the 
''learning by doing** approach in an environment that 
approximates the real world. 



Knowledge Transfer 

Successful simulations that approximate the real world 
setting have been developed for complex training where 
split second drxision nudcing can involve life and death 
situations. These simulations allow the learner to 
interact with real situations and immediately experience 
the effects of their actions without dire consequences. 
For example, medical students can learn to become 
proficient in medical diagnose through carefully selected 
case studies where the student selects a treaunent in 
response to physical symptoms and then observes the 
effect of his or her decision on the patient 

Other less dramatic simulations, called process 
simulations, are used to inform learners about a process 
or concept that does not manifest itself visibly. Such 
simulations can assist with the learning of subject 
matter in economics or genetics. In an instructional 
simulation called Xatlab**, the student can select 
physical characteristics of a male and female cat about 
to mate. The simulation speeds up the process and the 
student observes the resulting characteristics of th'^ 
kittens. Through repetitive activities with the 
simulator, the student eventually begins to understand 
how genetic laws of inheritance operate. 

Other simulators teach a sequence of actions that 
constitute a procedure. Some procedural simulations, 
such as the learning of cardiopulmonaiy resuscitation, 
may only have one correct sequence of steps. Other 
procedures, however, may involve one or more possible 
sequences of steps. The choice of the appropriate 
sequence depends on the use of higher level skills such 
as analysis, synthesis, and evaluation to determine the 
appropriate procedure. An example of this type of 
simulation would be the training of police officers to 
handle emergency family disputes. 

Still other simulations allow the trainee to explore the 
effects of his or her actions on the attitudes and 
behaviors of others in simulated on-the-job problem 
situations. An example of a situational simulation 
might be a management problem involving the use of 
different leadership styles. A company is forced to 
down-size its training depaitment, how will the manager 
handle the situation? Different leadership styles will 
result in the choice of different actions. In each case, 
feedback in the form of resulting outcomes will be 
given based on the choice of action. The advantage of 
this type of training is to allow the manager to 
experiment with different leadership styles without the 
penalty of failure. 

Summary 

Traditional training methods are rapidly becoming 
obsolete. New approaches and techniques are beginning 
to surface that promise improvement in the speed, 
effectiveness, and transfer of learning. Many of these 
transfer strategies involve a creative approach in that 
one must re-think the learning process and become more 
sensitive to the learning needs of the student New 
strategies call for an intuitive, creative uainer who can 
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design and ddvclop learning activiucs where there is 
cooperation and sharing between managers and trainers, 
between trainers and students, and between student and 
student. 

The emphasis for future training will be on ""being of 
service" to the learner. Effective learning will involve 
asking ihe student, "What experiences do you need to 
loam this content?" The emphasis will no longer be just 
on content, but rather on learning how to learn, learning 
how to ask good questions, and learning how to develop 
higher-level thinking skills. 



Trainers must begin to think of creative alternatives to 
old learning styles. Creativity will be essential in 
designing a blend of activities for the visual, aural, or 
the kinesthetic learner. Activities must be created that 
focus on the development of higher-level skills, 
problem solving, creativity, and metacognition. 
Training in the 1990's will offer an exciting challenge 
for those who are willing to go beyond traditional 
assumptions and old educational paradigms. 



PRE>TRAINING 



TRAINING 



POST^TRAINING 



TRAINER 

Meet with supervisor/manager to: 

• Identify desired performance 
skills. 

• Identify performance standard. 

• Identify supervisor's role in 
support of performing new skill. 

• Encourage super - to woric 
closely with erat \ c. 

• Familiarize superv.. jr with 
course content 

• Communicate with supervisor 
suggested methods for support, 
reinforcement* or role modeling. 

• Consider developing employee 
self-assessnoent instrument 

• Develop any pre-course 
assignment such as reading or 
researching of information. 

• Assess barriers to transfer in the 
woik environment (within 
employee, supervisor, or 
organization). 

SUPERVISOR 

• Memo to employee indicating 
managements interest in training 
as well as resulting on-the-job 
performance. 

• Discuss importance of training 
with employee., 

EMPLOYEE 

• Understand the organization's 
objective for training. 

• Preliminary reading and/or 
research on topic. 



TRAINER 

• Find out employee expectations 
and problem areas to be 
addressed. 

• Provide opportunities for 
interactive dialogue. 

• Provide opportunities for mental 
experimentation and creativity. 

• Provide cooperative learning 
opportunities. 

a. PrD\ide problem solving 
activities where trainees can 
observe how others draw 
conclusions and reach 
solutions. 

b. Provide exposure to 
alternative points of view. 

c. Debrief employee following 
activity. 

d. Provide feedback on 
correctness or incorrectness 
of thinking. 

• Use case-studies, role-play 
situations, simulations, or 
games. Be sure all activities are 
structured and have 
organizational relevance. 

• Have trainee develop an action 
plan for implementing new 
knowledge and skills. 

• Have trainee work on a real 
project during training. 

• Hold a discussion where trainees 
focus on problems they will 
have to apply new knowledge 
and skills to. Discuss anticipated 
implementation problems. 

• Provide opportunities for 
discovery learning 

• Test for understaiiding and 
application of principles. 



niAINER 

• Follow-up session where 
discussion centers on what was 
learned and how it will be 
applied to job. Include 
supervisor. 

• If the time between training and 
the use of the knowkdge and 
skills is greater than several 
weeks, then provide refresher 
session. 

• Hold a sesskMi lo discuss how an 
emptoyee apidied the new 
knowledge and skills. Discuss 
benefits and any problems. Use 
this employee as a resource to 
assist others with 
implementation. 

• Provide siq)ervisor with 
information on how to fulfill 
nrfe as coach and siq)pofter. 

• Develop job aids to assist 
employees with applkation of 
new skills. Be sure all 
perfonnance aids have been 
approved by the organization. 

• Post-training questionnaire to 
assess the strategies most 
effective in facilitating transfer. 

SUPERVISOR 

Reinforce and support new 
skills. 

Provide safe environment for 
trying out new skills. 
Hold employee rccountable for 
implementing his/her action 
plan. 

Provide feedback on employee's 
performance and suggest 
improvements. 

Provide opportunities for team 
support 
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DO ICONS IMPROVE THE USER INTERFACE? 



Carol B. MacKnight, University of Massachusetts 



Abstract 

The success of the Macintosh interface has brought with 
it a deluge of systems featuring iconic interfaces. The 
appeal of graphical representations often tends to make 
icon driven systems appear simpler to use than menu 
based intafaces. As the trend toward iconic interfaces 
grows, one is led to believe that direct accessibility of 
objects and actions on a disp?ay improves user 
performance. An exploratory study was conducted by the 
author to determine how successful users were in 
le:;ming and using both types of interfaces. Results 
indicate that the iconic interface has a positive effect on 
learner performance. 

introduction 

A rapidly growing number of users with little or no 
knowledge of a programming language are using 
authoring systems to facilitate the development of 
instructional materials. For this audience, difficulties in 
learning the system may become a permanent 
obstruction to their using the computer in a creative 
way (Solloway, ei al.,1982). It is said that the first few 
hours spent learning a system are the most important in 
determining further continuation by potential users 
(Dumais and Landauer, 1982). 

Using an authoring system should speed the authoring 
process, making it easier, faster, and cheaper to create 
instructional programs by eliminating the need to hire a 
professional programmer or learn a programming 
language (MacKnight and Balagopalan, 1988-89). 
Since specification depends on recognition and not recall 
(Dumais and Landauer, 1982), both menu based and 
icon driven authoring system are not supposed to either 
require special training or an extensive learning time. 
For that matter, any system designed for users should be 
easy to learn and use, pleasant to use, and contain Cie 
functions users need in their work (Gould and Lewis, 
1983). 

But Cook and Philipps (1989) found this not to be the 
case with iconic systems. They describe an excessive 
learning time required before anything useful can be 
accomplished as a major drawback of icon systems. 
Moreover, "lesson functions, " they say, "which are 
normal part of most lessons may be considerably 
difficult if not impossible to implement using icon- 
oriented systems." 

Our focus is on how new users interact with menu and 
icon driven authoring systems. Menus are simply lists 
of options available to a user in a tree structure, while 
options under icon driven authoring systems are ever 
present on the display m the form of icons and pull- 



down menus. On the face of it, this major difference 
might suggest that a system which provides direct 
accessibility is easier to learn and use. Our interest lies 
in discovering what difficulties new users may have in 
using these system and determining the source of that 
difficulty by examining the errors users make. 

In this paper, we present the results of an exploratory 
study in which new users with programming knowledge 
were asked to create a small instructional program. Our 
objectives were to examine the errors that new users 
make and determine if there are significant differences 
between interfaces. Such a study will have implications 
for the improvement of training and self-instructional 
materials. 

Design of Experiment 

Ten students were recruited as volunteers from the 
Department of Computers and Electrical Engineering 
and from the Computing Services Staff of the 
University of Massachusetts and tested individually. 
Eight students, ranging in age from 19 to 30 years old, 
were able to complete both weeks of testing before the 
si*mmer break. The students had taken from one to 
seven programming courses for credit One student had 
taken one credit course, tour had taken two courses and 
the other three had taken three, four, and seven courses 
respectively. Four of these students fell that they had a 
stronger Macintosh orientation, two said tney were 
equally comfortable on either machine and two worked 
primarily on the IBM. 

Learners were blocked, according to their rating of their 
own strengths in using either a Macintosh or an IBM 
personal computer on a pre-session questionnaire. Then, 
they were randomly assigned to begin the^r learning 
experience on one or the other machines. Data was 
tabulated only for the eight students who completed 
both tests. This left four Macintosh users and four IBM 
users two of whom were equally adept on the 
Macintosh. The authoring systems used were an icon 
driven system which resides only on the Macintosh and 
a menu based system that runs only on the IBM. These 
systems were selected because their similarities in 
functionality permitted us to map our instructional 
lesson onto the set of tools that each provides. 

Each learner received one and a half hours of self- 
teaching followed by a testing period of equal length. 
The self-teaching consisted of familiarizing oneself v iih 
a description o^ the computer control keys, the system's 
hierarchical structure, functionality of icons or menu 
objects and options, and a set of six step-by-step 
practice exercises designed to teach learners the 
operations involved under six of ten funcuonal areas in 
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authoring courseware — authoring environment, text 
creation and editing, graphics editing, animation and 
instructional strategy(For an in depth description of 
functional areas and related tasks see MacKnight and 
Balagopalan, 1989.) These exercises were identical in 
functions and tasks to those presented in the testing 
session for each system. Learners were encouraged to 
complete all exercises before repeating any. At the 
conclusion of the practice session, learners were given a 
IS minute break before beginning the testing session. 

The four test exercises, which were identical tor each 
authoring system, included functional areas and tasks 
frequently performed by authors in developing 
instructional programs. The first exercise, a cover page, 
measured ability to import a graphic and use the correct 
style, font, size of text, and placement The second 
exercise was a menu with three items, two of which 
branched to two other screens and tiie third branched to 
the next exercise. The third exercise contained a four 
item multiple choice question with feedback and 
branching. The fourth exercise* an animation exercise, 
required importing a graphic and animating the text 

These exercises measure whether learners understood die 
function of icons or menu objects and options and 
mastered the operations practiced in the self- 
instructk>nal session. Each learner was presented with a 
script and flowchart of a mock lesson. Since we were 
not interested in typing skill» typing was kept to a 
minimum and the use of color was not introduced. The 
challenge was to produce a brief introductory screen 
which required importing a graphic* followed by a menu 
screen, a multiple choice screen, and a closing screen 
which required an animation, and have the mini lesson 
function from beginning to end. 

Learners were asked to think-aloud, when they were 
unsure of what to do next Whenever there was silence 
during a pause of IS seconds or longer, the observer 
asked the learner to verbalize the problem. Upon 
completing the test, learners were asked to evaluate their 
own performance in an exit interview and complete an 
exit questionnaire. One week later, learners repeated the 
entire procedure, switching authoring systems and using 
the same exercises, which were now randomly ordered. 
In this study, the total time that each learner spent in 
practicing and testing on the two systems was 
approximately six hours. 

Data Collection ar.d Measurement 

Data was gathered from a pre-session and two post- 
session questionnaires — one for each authoring system 
and a filial session interview. Learners were asked to 
ignore the observer, who sat in the room during the 
practice and testing sessions. The basic function of the 
observer was to help if the system went down, to note 
points of error, which would facilitate documenting 
errors on the video tape, and to see that learners 
verbalized any difficulties that they were experiencing. 
Errors are defined as either failing to select or 



inappropriately selecting a menu item con, cursor, 
text tool, text font, text style, text size, screen location, 
content location, screen erase, erase feedback, erase 
answer, multiple choice format, performance parameters 
options, object to animate, direction of animation, 
graphic, branch, and making invalid entries. 

Results and Discussion 

A srnali scale ^'.xploratory study was designed to 
determine whether or not new learners with a 
programming background would be able to distinguish 
the appropriate context in which to use icons or menu 
prompts when designing an instructional lesson. 
Questions asked were: 

Will new learners, using an icon driven 
authoring system, do better in comprehension, 
debugging, or modification than learners using 
a menu ^ ased authcmng system? 
2. Will learner satisfaction be significantly 
different between those using icon and menu 
authoring systems? 

^^Its were recorded through objective performance 
n. ''sures, session videotapes, questionnaires and 
interviews. Each individual's total score for week one 
was subtracted from his or her total score received in 
week two over all factors (total time excluding 
debugging time, total time including debugging, total 
errors transcribed from the videotape excluding 
debugging, and total errors in the hard copy of their 
program after debugging.) A Mann Whitney U Test was 
used for comparing scores. 

As Table I and Table 2 indicate the effect of interface 
style was significant at the p <.05 level of confidence. 
When debugging time was subtracted, iconic system 
learners out performed menu system learners, spending 
less time and making fewer errors in completing the 
exercises. 

These results are in contrast with those of Whiteside and 
colleagues (1985). In comparing seven user interfaces 
on a benchmark file manipulation task, they found that 
new users performed better on the command and menu 
system than they did on iconic systems. For system 
users, there were no statistical significant differences 
between iconic systems and other systems. However, 
user performance was based strictly on rate of task 
completion. It is interesting to note that when we 
applied their formula to our time data before debugging, 
our results still obtain, contradicting their findings. 

Other factor, such as total time, including time spent 
debugging the exercises and errors detected in the hard 
copy of their program after debugging, did not reached 
statistical significance. The pattern of differences 
indicated a slightly lower score for the icon over the 
menu group. See Tables 3 and 4. 
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Table 1. Tinfe in minutes to complete all exercise 



without debugging. 



Subjects 


Menu 


Icon 


Change Sc 


1 


73 


50 


23 


2 


72 


42 


30 


3 


35 


25 


10 


1 4 


59 


25 


34 


5 


40 


38 


2 


6 


84 


49 


35 


7 


49 


28 


21 


8 


59 


41 


18 



Mann-Whitney U test of difference between menu and 
icon: Menu Group = 4 JO; Icon Group = 0.00; U = 
0.00:z = '2J205:p = .0ll7. 



Table 2> Total errors made in all exercises on videot ape. 



Subjects 


Menu 


Icon 


Change Scores 


1 


56 


42 


14 


2 


59 


32 


27 


3 


28 


21 


-7 


4 


76 


16 


60 


5 


45 


28 


17 


6 


73 


35 


38 


7 


44 


43 


1 


8 


49 


55 


-6 



Mann-Whitney U test of difference between menu and 
icon: Menu Group = 4.86: Icon Group = 2 JO; U = 
0.00; z = 2.2404; p^, 0251 



Table 3. Time in minutes to complete all exercise 



with debuggin g. 



Subjects 


Menu 


Icon 


Change Scores 


1 


76 


96 


-20 


2 


72 


42 


30 


3 


41 


44 


-3 


4 


59 


35 


24 


5 


91 


71 


20 


6 


93 


83 


10 


7 


60 


43 


17 


8 


^2 


81 


-19 



Mann-Whitney U test of difference between menu and 
icon: Menu Group = 5.70; Icon Group = 3.50: U = 
0.00; z ^ 1.0502; p = .2936 two-tailed. 



Table 4. Errors made in program hard copy after 



debugging. 



Subjects 


Menu 


Icon 


Change Scores 


1 


28 


11 


17 


2 


28 


28 


0 


3 


16 


8 


8 


4 


42 


n 


31 


5 


12 


08 


4 


6 


29 


06 


23 


7 


13 


07 


6 


8 


23 


30 


-7 



Mann-Whitney U test of difference between menu and 
icon: Menu Group = 4.17; Icon Group = 3.00; U = 
0.00; z = -1.859; p = 0630 two-tailed 



Both interface style and order of interface style 
introduction impacted on time needed to complete the 
exercises, and the number of errors made before 
debugging. In general, new users found the icon 
interface easier to use than the menu based system. 
There are several possible explanations for these 
fmdings. The graphical user interface of the icon system 
allowed users to know where they were, what options 
were available, and to quickly correct a wrong choice. In 
contrast, the menu driven system required more back and 
forth movement among menus and more steps to 
complete tasks-all of which added to the total time 
spent in completing the instructional lesson and 
increased the chances of making errors. 

V/hen debugging time was added to the total time, the 
small difference between interface styles was 
insignificant. There were many possible contributing 
factors for this fmding. Hrst» the icon authoring system 
contains a better debugger than the menu based system 
and people used it In contrast, tvyo pe<^le simply did 
not use the debugging facility on the menu driven 
system. Second, the learner approach to the software 
was distinctively different. On the Mac* itosh the 
learning approach was more by trial and error with the 
learner engaged in debugging almost from the start 
This type of approach* however, is inappropriate (or 
learning a menu based system where the Iczivcr risks 
getting lost in the difre<^t levels of menus. 

The results of the exit questionnaires administered at the 
conclusion of each testing session produced no 
difference between groups in terms of learner 
satisfaction. However, there were differences reported in 
ease of use and difficulty experienced in using the 
systems (See Tables 5 and 6). 



Table S> Difficulty in using the system. 





Menu 


Icon 


Definitely Agree 


2 


3 


Inclined to Agree 


4 


2 


Inclined to Disagree 


1 


3 


Definitely Disagree 


1 


0 


Sample Size = 8 






Table 6. 1 learned this system easily. 






Menu 


Icon 


Yes 


7 


3 


No 


1 


5 



Approximately three fourths (6/8) of the menu and icon 
learners said that they learned the authoring system 
easily. But 7/8 said they had difficulty in using the 
menu system, while only 3/8 said they had trouble in 
using the icon driven system. Their perceptions 
correlate well with their performance measures, thus 
lending some additional support. 

Some of the difficulties learners mentioned in using the 
menu driven system were related to editing text, lack of 
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good error detection, graphics manipulation, branching, 
and links between menus. Icon learners listed these 
items as those being the cause of certain difficulties: 
confusing submenus , icons taking up too much screen 
space, lack of on-line help, lack of error messages, the 
different erase options and not fully understanding the 
wait and erase icons and the function of the map icon. A 
detailed account of errors will be reported elsewhere. 

Summary 

If new usen are to continue using an authoring system, 
they must be able to achieve a level of learning that is 
encouraging. Often the practice examples in manuals 
fail to bridge the gap between what the authCH* of the 
manual thinks a person must ieam, and in what order, 
and what the new user needs to know in order to 
survive. Frequently, manuals are roost useful after the 
user has some degree of proficiency in using the 
system. To facilitate learning, we need to know where 
learners make errors and why. With this understanding 
we can begin to orchestrate training programs and self- 
instructional materials that make a difference. 

In this paper we have examined whether a style of 
interface inakes a difference in user performance. We 
found that it does. We have learned only that the menu 
style of interface presents more of a challenge for new 
users to learn and use. Gearly, we don't know whether 
this will hold true over a longer period of time, with 
different users, or when lesson size dramatically 
increases. 

From this study, we have, however, learned that it is 
possible to train students to perform tasks that are 
important parts of instructional lessons in a much 
shorter time than in the past Constructing detailed 
scripts, showing hovv key tasks should be performed 
with the new authoring system, proved to be a 
successful method of learning the system. 



Acknowledgements. Special thanks to ^iung-yao 
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MICROCOMPUTER-BASED INSTRUCTIONAL SIMULATION 
FOR TRAINING RESEARCH 



Joseph S. Mattoon and Richard A. Thurman, Air Force Human Resources Laboratory 



Microcomputer-based instruciional simulation (MBIS) 
is becoming more valuable for training and training 
research in military and civilian settings. In the last two 
decades, advances in hardware and software technology 
have made it possible to develop high-fidelity 
simulators (e. g., pilot training simulators that 
resemble a real aircraft and provide complex motion and 
visual cues). However, hi'^h-fidelity simulation, which 
requires mainframe computers and large, expensive 
facilities, are usually designed for the purpose of 
precisely reproducing the system being simulated (e. g., 
F- 16 aircraft) rather than for training and research needs. 
The nature of the hardware and software components of 
high-fidelity simulators make them somewhat 
impractical to use as training research vehicles. The 
costs encountered in changing system configurations to 
produce only a few experimental treatments can easily 
outweigh the value of knowledge that is expected to be 
gained in conducting the research. Microcomputers are 
usually incapable of producing high-fidelity 
simulations, but they are often a better choice for 
conducting training research than mainframe simulators 
because of their low cost and flexibility for 
manipulating instructional variables and data capture 
procedures. Simulations are used for various types of 
training (e. g., transportation systems, nuclear reactor 
control rooms, and many other complex systems). 
Although this report will discuss advantages of MBIS 
for aircrew training, the theoretical aspects and 
conclusions are gencralizable to other training contexts. 

The term, simulator, refers to a computer-based system 
that is dedicated to simulating a particular device and its 
environment. The term, MBIS, refers to a 
microcomputer program without special hardware that 
can be developed using general purpose programming 
languages or hypermedia software. Simulators driven by 
mainframe computers usually have a bi^h degree of 
fidelity, while MBIS has a somewhat lower Jc^el of 
fidelity. The importance and degree of simulation 
fidelity to training complex skills such as piloting an 
aircraft is still poorly understood. Several types of 
fidelity have been identified (e. g., physical, functional, 
& psychological). Generally speaking, fidelity refers to 
the degree that a simulation faithfully reproduces or 
represents the way a particular phenomena would occur 
in the real world. For example, full-mission simulators 
used in Air Force pilot training are very much like real 
aircraft. The student pilot climbs into the simulator 
cockpit, lakes off, performs various flying maneuvers, 
and lands. The student uses almost exactly the same 
knowledge and skills s/he wouM employ when piloting 
the real aircraft even though the simulator remains 
mounted to the floor throughout the training activity. 
Although the view displayed in the cockpit window is 



obviously not that of the real world, and some aspects 
of flight are missing (e. g., G-force), the motion and 
visual cues are quite convincing and make students 
"feer as though they are actually flying. For this 
reason, full-mission simulators are referred to as higl>- 
fidelity simulatCM^. 

Unfortunately, full-mission simulators are not effective 
training devices for teaching entry level skills (Andrews, 
1988; Boreham, 1985). Entry-level learners can be 
overwhelmed by the complexity of piloting an aircraft 
due to the many instruments to monitor and the large 
number of decisions to make in a relatively shoit time. 
Microcomputers present a much less intimidating 
interface with the training activity for entry level 
learners. Low-fidelity MBIS represents a radical 
departure from the real worid task but have been shown 
to be more effective than high-fidelity simulators in 
some situations (Gagn6, 1962). Teaching student pilots 
how to operate individual instruments is often more 
effective than exposing them to the entire aircraft at the 
beginning stages of training. High-fidelity simulators 
are good for helping advanced learners 
integrate/consolidate multiple skills/tasks but are poorly 
equipped to isolate and train individual skills. 
Microcomputers can be programmed to simulate various 
aviation instruments and decision-making tasks and 
therefore, are quite effective for leaching entry-level 
skills. MBIS also provides an additional opportunity to 
deliver instructional treatments and collect performance 
data for training research purposes. For example, MBIS 
for Headup and Radar Electro-Optical displays, which 
aie used for navigation and weapons systems, have 
proven to be effective research and training tools. One 
important advantage of MBIS is its "stand-alone" 
qualuies. Unlike full-mission simulator training, 
students can work at improving their skills wiJiout the 
assistance of a simulator operator. Also, because MBIS 
is based on software rather than hardware components, it 
can easily be duplicated and run on microcomputers at 
other locations. This eases scheduling problems 
commonly encountered when setting up training and 
research activities. Finally, software can be altered to 
produce many different types of instructional treatments 
and capture various kinds of data on floppy or hard 
disks. Because treatments are designed to operate on a 
stand-alone basis, different experimental groups or even 
different experiments can be run simultaneously. 

Reigeluih and Schwartz (1989) state that "...few 
empirically-based prescriptions have been offered to 
guide the design of instruciional simulations." MBIS 
should not be confused with traditional types of 
computer-based training (CBT). CBT typically involves 
instructional materials that are similar to written 
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materials (e. g., presentation of learning objectives, 
content, practice, and multiple*choice tests) except for 
the additional branching capabilities made possible by 
the computer. Although much research has been done 
using the CBT paradigm, in comparison, there have 
been very few experimental studies that have examined 
real-time information processing in the dynamic 
environment of training simulations. MBIS provides 
instructional researchers with this capability. 



Advantages of MBIS Over High-Fid 


elity Simulators 


Characteristics 
& Concerns for 
Cbnducting 
Training 
Research 


High-Rdelity 
Simulators 


Low-Fidelity 
MBIS 


Cost of 
haidwarc. 


very high 


very low 


Cost of 
software. 


very high 


low 


Portability of 
experiments to 
other locations 
&acn>ss 
systems. 


very low 


high 


Generalizalnlity 
of cxp. results 
toother training 
environments. 


low 


high 


Capability to 
isolate & 
manipulate 
instructional 
variables & 
factors that 
affect learning & 
perfomiance. 


low 


high 



The use of MBIS to conduct training research will 
provide an q)portunity to examine some important 
factors that affect the acquisition and retention of 
knowledge and skills. Simulations have some gaming 
characteristics that are highly motivating (Orbach, 
1979; Ruben & Lederman, 1982). Learners receive 
infomiation and instruction as active participants in the 
attempt to control a particular system or environment, 
and this can be an exciting and enjoyable challenge. 
Furtlier research needs to be done to identify the factors 
that make games and simulations fun and facilitate the 
acquisition of new skills as a function of learners* 
intrinsic motivation for learning tasks (Deci, & Ryan, 
1980; Malone, 1981). Also, strategies used by learners 



to attend to and encode visual, aural, and symbolic cues 
need to be explained. Alessi (1988) has suggested that 
there are optimal levels of simulation fidelity in terms 
of cost and training effectiveness. Such parameters 
apparently vary as a function of learners* prior 
knowledge and the type of task to be learned. The table 
shown below summarizes the advantages of MBIS over 
high-fidelity simulators for conducting training research. 
It is the opinion of the authors that a better 
understanding of fidelity and other variables that 
moderate instructional effectiveness of simulations can 
be attained efficiently through research using MBIS. 
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EFFECTS OF ADJUNCT QUESTIONS ON LEARNING FROM HYPERTEXT 



Wayne A. Nelson, Southern Illinois University at Edwardsville 



Abstract 

An experiment was conducted to examine whether 
adjunct questions in various configurations within a 
hypertext document had any effect on retention of 
information. Results indicated that there were 
significant differences in retention, with the greatest 
retention produced by a Read Only treatment with no 
adjunct questions. Various factors which could account 
for these findings are discussed. 

Recent claims that hypertext will revolutionize the 
ways that people read and wnte make the technology 
appealing to educators (Conklin, 1987; Kerr, 1988; 
Marchionini & Shneiderman, 1988). Hypertext allows 
direct interaction wifh information, leading to rapid and 
efficient access to further, related information. Readers 
have unprecedented control over the scope and sequence 
of what they read, which requires a pragmatic approach 
to reading based on exploration and conjecture (Stevens, 
1986). It has been suggested that hypertext also 
supports a mode of learning which differs significantly 
from frame-based computer-assisted instruction (CAI) 
and model-driven intelligent tutoring systems 
(Hamnnond, 1989). To be sure, hypertext is not a 
panacea for education. There is no guarantee that 
exposure to vast amounts of information will enhance 
learning or ensure understanding any more than a library 
docs. But there is some evidence that hypertext 
environments may be used to create better educational 
software (Jonassen, 1988; Yankelovich, Mcyrowilz, & 
van Dam, 1985). 

While hypertext applications for education are 
blossoming, research focusing on the efficacy of the 
technology has not kept pace with development. Users 
encounter many problems in using hypertext, including 
getting lost, missing important information due to a 
poor understanding of document structure, inefficient 
search strategies and conflicts between the user interface 
and the task donain (Conklin, 1987; Hammond, 1989; 
Jones, 1989; Nelson, 1990; Shneiderman, 1989). Some 
of these problems are mitigated by creative design of 
hypertext systems which takes into account the need for 
learner guidance in specific activities and access 
strategics (e. g., the Intermedia project described by 
Yankelovich, Meyrowilz, & van Dam, 1985 ). But 
research examining the underlying cognitive processes 
involved u: reading hypertext documents has been 
minimal 

If we assume that reading is reading, regardless of the 
medium, then the processes of learning from prose 
should be similar for hypertext as compared to standard 
textbooks. A large body of research has examined 



various instructional techniques which make prose nore 
understandable. One technique which has been shown to 
be effective is the use of adjunct questions (Rothkopf, 
1966; Rothkopf & Bisbicos, 1967). When learners read 
a passage with the addition of adjunct questions, 
performance on tests of retention is better than for 
learners who did not read the adjunct questions. Studies 
have examined both the placement and the types of 
adjunct questions. Rothkopf (1966) and Boker (1974) 
found that both pre questions and post questions 
enhanced learning, but pre questions tended to inhibit 
"incidental" learning because the questions cued the 
readcr*s attention to information related to the pre 
questions. Rickards and DiVesta (1974) found that 
simple rote questions resulted in memorization of 
unrelated facts, while meaningful questions encouraged 
the learnCiS to organize material into coherent 
structures. Mayer (1975) found that performance on a 
post test of retention was correlated with the type of 
questions embedded within the text. Learners who read 
passages with adjunct questions requiring a higher level 
of cognitive processing also performed well on a test of 
all other types of questions. 

If the placement of adjunct questions are influential in 
determining what is learned from prose, will the same 
results occur for hypertext documents? The study 
reported below examines this question in terms of 
performance on retention tests. It is hypothesized that 
results will be similar to previous studies of traditional 
text, in terms of retention in both intentional and 
incidental leamirig. Pre questions should promote better 
intentional learning, while post questions should 
pomote better incidental learning. 

The results of this study may lead to a better 
understanding of the cognitive processes involved in 
learning from hypertext, and may help iO clarify some 
of the issues facing designers of hyj/ertext systems. 
More complete information regardirg the cognitive 
processes and instructional strategies requlicu ic use 
hypertext systems is needed. If the fnocesses and 
strategies are similar to traditional text, then we may 
proceed in the design of hypertext learning 
environments using our current knowiodge of how 
people learn from prose. But if discrepancitc in learner 
performance are discovered with hypertext document:;, it 
may be necessary to further examine the differences 
between hypertext and traditional text, and revise our 
instructional strategies accordingly. 

Method 

Participants. Thirty six participants were randomly 
selected from a pool of volunteers in educational 
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Figure 1* Experimental Design. 



Immediate 
Retention Time 
Delayed 



Adjunct Questions in HyperTcxt 



Experimental Treatments 

Pre Post 
Read Only Questions Questions 

Retention Type 
(Intentional/Incidental) 



Embedded 
Questions 



1 
1 


1 
1 


1 
1 


1 
1 


1 
1 


1 
1 


1 
1 


1 
1 



psychology classes at a midwestern university. 
Although participation was voluntary, subjects received 
partial course credit for their involvement in the 
experiment. The participants were familiar with 
computers in general (i. e., had used word processors, 
etc.), but had no experiv nee with hyp^xt documents. 

Design. Because reading hypertext may differ from 
reading traditional text, due to having to learn both the 
navigation procedures of the system as well as the 
content, some control was needed to account for 
participants* initial unfamiliarity with a hypertext 
system, and to assess the effects of adjunct questions as 
participants gained expertise in using the hypertext 
system. Therefore, a 4x2x2 randomized block design 
with repeated measures on all factors was implemented 
in order to examine the effects of growing familiarity 
with the hypertext syst Ti (See Figure 1). Participants 
each received all four experimental treatments in a 
random order. The experimental treatments represented 
different adjunct quesdon strategies (pre questions, post 
questions, embedded questions and read only) which 
were employed in a hypertext document. The Pre 
Question group viewed four questions prior to reading 
the hypertext section to which the questions related. The 
Post Question group viewed four questions after reading 
the related hypertext section. The Embedded Question 
group viewed four questions interspersed within the text 
of the related hypertext section. The Read Only group 
simply read the material without viewing any questions. 

Participants completca four sessions, one for eacii 
experimental treatment. The other repeated measure 
factors were retention time and retention type. Retention 
time represented immediate and delayed t ^tention tests, 
while retention type represented int'.*ntional and 
incidental retention of information from the hypertext 
documents. 

Materials and Equipment. Our hypertext 
documents which presented concepts associated with 
educational psychology were developed for use in the 
experimenL The documents, which ranged from 1 1 10 to 
1250 words in length, contained information regarding 
various aspects of cognitive and behavioral learning 



theories, cognitive development, measurement and 
evaluation. The information was adapted from standard 
undergraduate educational psychology texts (Dembo, 
1988; Mayer, 1987), and was delivered using HyperCard 
on the Macintosh computer. 

Information in the documents could be accessed through 
associative links contained within the text of each 
screen, with additional links available to a help screen 
and a full-screen map. Additional navigation aids 
included a backtracking facility, along with a miniature 
"map" of the document structure showing the overall 
organization of the document and the information the 
user had already viewed (See Figure 2). The sc ftware 
was programmed in order to collect data regarding the 
time spent on each screen and the sequence of screens 
accessed by each participant 

One set of eight short-answer retention questions were 
developed for each of the four documents. Each set of 
questions contained four intentional questions and four 
incidental questions. The intentional questions were 
presented in the hypertext documents and were viewed 
by the participants in pre, post, Oi embedded 
configurations. The incidental questions, based on the 
content of the hypertext docume nts, were not seen by 
the participants until the retention test was given. The 
sets of retention questions were arranged in a random 
sequence for presentation to the participants. 

Procedure* Participants spent a total of four hours 
reading the documents over the course of four weeks, in 
one-hour sessions each week. The sessions were 
conducted with groups of fifteen participants, each using 
their own computer.. At the first session, participants 
were given instructions and training on the use of dte 
computer and a small hypertext system (identical in 
design to the experimental system, but with different 
content). When they were comfortable with the system, 
they were informed of the nature of the experiment and 
given instructions to study the material in preparation 
for a test. During each session, an experimenter was 
available to answer questions about the operation of the 
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Figure 2. Screen design and navigation aids included in hypertext documents. 



General Guidelines for Teaching 



In gextfral, a cognitive viev of teaching portrays the student as 
active processors of irrfonnation. Cognitively oriented teachers 
recognize the importance of determining vhat their students 
knov and basing instruction on students' levels of knovledge so 
that basic cognitive processes are supported. A cognitive 
&;^proach to teaching also is based on the ideas that students learn 
best if materials are meaningful, so that Iiiglier levels of 
cognitive procesjing are supported. Finally, a cognitive 
orientation recognizes that teachers need to consider 
individual differences in cogrutive abilities vhen planning 
instruction. ^ ' 
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system, but no information related to the content was 
given. Participants read at their own rates, then 
completed an immediate test for retention. The same 
retention test was completed by each participant one 
week later in order to measure delayed retention. 

Results and Discussion 

Data included the scores on immediate and delayed 
retention tests, the amount of time spent reading the 
documents, and the total number of screens accessed. 
Analysis of variance (ANOVA) procedures with repeated 
measures were employed to determine whether 
differences in mean scores on immediate and delayed 
tests could be attributed to differences in placement of 
adjunct questions, if there were any differences in recall 
of intentional or incidental information, if the total time 
spent reading varied significantly between groups, and 
whether the amount of information accessed by 
participants differed significantly. Analysis of the 
retention scores was completed in two phases. A key 
was consu-uctcd for the questions, and an experienced 
rater then compared the ideas from each participant's test 
to the key, giving one point for each idea which was 
found in the recall key. A second rater completed a 
randomly selected subset of the tests using the same 
scori»]g procedures. Reliability between the two raters 
was .91. 

Mean total scores for the immediate and delayed 
retention tests are given in Table 1. A 2(immediate or 
delayed retention) x 4(toial retention score for each 
treatment) ANOVA with repeated measures showed no 



significant differences between immediate and delayed 
retenuon (F(l,68)=2.277, /k0.136) and no significant 
interaction (F(3,68)=.816, p<.4863), but there were 
significant differences for retention between treatments 
(F(3,68)=2.77, p<.0427). Multiple comparisons usl.ig 
Fischer's LSD pro<.edure revealed that the Read Only 
treaunent produceri the greatest delayed retention, an'^ 
that the Post Questions treatment produced significantly 
greater delayed retention than the remaining adjunct 
question treatments.. There were no significJirt 
differences in immediate retention be! /een the 
treatments. 

Tables 2 and 3 present the intentional and incidental 
retention scores for each treatment A 2(immediate or 
delayed retention) x 2 (incidental or intentional) x 4 
(treatment type) ANOVA with repeated measures 
showed that retention of intentional and incidental 
information was significantly different (F(l,140>:7.247. 
p<.008), but was not influenced significantly by the 
expcrir-nlal treatment (F(3,420)=2.33,/k.0735), or by 
the type of retention (F(l,140)=1.331, p<.2506). 
Multiple comparisons using Fischer's LSD procedure 
revealed that participants recalled more intentional 
information than incidental information. Similar 
comparisons showed that the Read Only treatment 
produced significantly greater retention on intentional 
questions for the delayed test than for the immediate 
test. This difference may have produced the interaction 
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Table 1. Mean total retention scores for immediate and delayed measures. 









ExDerimental Treatments 








Read 


Only 


Pre Questions 


Post Questions 


Embedded 


Questions 




Mean 


SD 


Mean 


SD 


Mean 


SD 


Mean 


SD 


Immediate (N=36) 


4.40 


1.91 


4.11 


1.87 


4.11 


1.89 


3.91 


1.95 


Delayed (N=35) 


4.03 


1.62 


3.23 


1.64 


3.91 


1.74 


3.43 


1.58 



Table 2. Mean retention scores for immediate and delayed Intentional questions . 





Experimental Treatments 




Read Only 


Pre Questions 


Post Questions 


Embedded Questions 




Mean 


SD 


Mean 


SD 


Mean 


SD 


Mean 


SD 


Immediate (N=36) 


2.31 


.95 


2.11 


1.19 


2.28 


1.08 


2.19 


1.14 


Delayed (N=36) 


2.56 


.94 


1.83 


1.16 


2.08 


.94 


1.94 


1.09 



Table 3. Mean retention scores for immediate and delayed incidental questions. 





Experimental Treatments 




Readonly 


Pre Questions 


Post Questions 


Embedded Questions 




Mean 


SD 


Mean 


SD 


Mean 


SD 


Mean 


SD 


bnmediate (N=36) 


1.75 


1.13 


2.03 


1.05 


1.94 


1.12 


1.83 


1.18 


Delayed (N=36) 


1.89 


1.14 


1.53 


1.03 


1.92 


1.23 


1.64 


1.02 



between retention type and treatment (F(3,420)=2.991, 
p<.0308) which was revealed by the ANOVA. The 
amount of time spent reading the hypertext documents 
was not significantly different between treatments 
(F(3,68)=L699, p<.1763), however, there was a 
significant difference in the amount of information 
accessed in the different treatments (F(3,68)=4.625, 
/X.0055). Multiple comparisons using Fischer's LSD 
procedure showed that participants in the Pre Questions 
treatment viewed significantly less information than the 
oib'^r treatments. These results are consistent with the 
findings of Rothkopf (1966) and Boker (1974), 
indicating that pre questions^ tended to inhibit careful 
reading of all information in the hypertext documents in 
favor of a focus on information that was related to the 
pre questions. 

In summary, significant differences in retention of 
information presented in hypertext documents were 
found, with greater retention for the Read Only 
treatment than for treatments which included adjunct 
questions in various configurations. These results are 
surprising when compared to similar studies of the 
effects of adjunct questions in text Differences found in 
the present study may be due to several factors. First, 
there is the issue of whether or not adjunct questions are 
effective for learning from prose, even in traditional text 
(Lindner & Rickards, 1985). Although effects were 
found in the original studies (Rothkopf, 1966; 
Rothkopf & Bisbicos, 1967), those experiments were 
performed using short passages (SO to 100 words) of 
unfamiliar information. The information presented in 
the current study using hypertext documents was at least 
somewhat familiar and meaningful to the participants, 
and prior knowledge may have confounded the results of 
this study. Second, four trials may not be sufficient 
time for participants to become comfortable and 



efficient in reading hypertext documents, and the adjunct 
questions may have influenced their reading habits. This 
is especially evident with regard to the amount of 
information accessed in the different treatments. Third, 
the length of the documents may have contributed to the 
results of this study. There may have been too much 
information to read, which interfered with the 
effectiveness of the adjunct questions, as indicated by 
the superior retention of the Read Only treatment 

Conclusions 

The results of this preliminary study do not rule out the 
use of adjunct questions in hypertext documents, 
however, special care should be taken in the placement 
and number of adjunct questions in hypertext Problems 
could arise, especially if adjunct questions are used in a 
pre question configuration. It may be that adjunct 
questions could reduce the sense of exploration that is 
often an important feature of hypertext Further research 
is needed to determine some of the other effects of 
adjunct questions in hypertext documents. This research 
should focus initially on the placement, number and 
type of adjunct questions. Since there is no way to 
predict the sequence in which information will be read, 
placement of adjunct questions becomes especialW 
difficult. It may not be effective to place adjunct 
questions on every screen, and the optimal number of 
adjunct questions per words of text needs to be 
determined. 

Alternatives to adjunct questions and other 
m':ithemagenic strategies also need to be explored. It is 
likely that constructing a hypertext document will result 
in better retention of information than simply reading a 
hypertext document Other strategies, such as note- 
taking, summarizing, organization, or constructing 
links should also be investigated. If we are to realize the 
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projected benefits of hypertext in education, we must 
investigate all possibilities for using hypertext in 
various instructional situations. 
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THE EVALUATION OF INTELLIGENT TUTORING SYSTEMS: 
A PLAN AND AN EXAMPLE 



Michael A. Orey, The University of Georgia 



Abstract 

This paper explores the realm of cvi luaiion within the 
domain of InicHigcni Tutoring Syste ns (ITS). Toward 
that end, Lillman and Soloway (1988) describe two 
facets of evaluation that arc examined: external ard 
internal evaluation. Described here arc a variety of 
procedures that they did not address and some t^at they 
discounted. One of the procedures not explored is the 
use of Hlocm's (1984) "2-Sigma Problem". Anderson, 
Boyle and Reiser (1985) used this procedure lo 
externally evaluate the Lisp Tutor. Priest and Young 
(1988) offer a variety of procedures that can be used for 
internal evaluation. These procedures are presented and 
POSIT (an ITS for subtraction of whole numbers) is 
used as an example case to illustrate the evaluation 
procedures. 

There has been a great deal of expectation for the 
benefits of Intelligent Tutoring Systems (ITS), but 
there is some equivocating evidence (Slasz, 1988) which 
does not support large scale expenditures for their 
implementation. However, i^earchers are beginning to 
address the issue of evaluating ITS (Anderson, Boyle, & 
Reiser. 1985; Liitman & Soloway, 1988; Priest & 
Young 1988). Anderson, el al., (1985) used an approach 
which pits their ITS against human tutors and a no 
tutoring condition. Littman and Soloway (1988) 
suggest that evaluation of ITS has two facets: extcmal 
and internal evaluation. Priest and Young (1988) present 
four different ways o. evaluating the student model 
within an ITSI argue here that Anderson's, et al., 
approach can be cast as one example of external 
evaluation while Priest and Young's evaluation is an 
approach to internal evaluation. I describe here these 
approaches and use POSIT (Orey & Burton, 1990) as a 
test ITS for the evaluation process. Before I begin the 
description of this evaluation procedure, it will help to 
describe the ITS used Ko demonstra e the procedure. In 
addition, a short description of tlu data collection is 
provided 

Description of POSIT 

POSIT is an ITS for the tutorin'j of whole-number 
subtraction. The design of this system (see, Orey & 
Burton, 1990, for a detailed description of the design) is 
based somewhat on Anderson'? (1987) ACT* theory of 
learning. Within this theoretical orientation, it is 
assumed that the learning of a cognitive skill builds 
from declarative knowledge. In terms of subtraction, the 
algorithm to be learned is made of a set of declarative 
information. During the development of this skill, this 
knowledge becomes consolidated into the specific skill 
(in this c . ^proficiency with a subtraction algorithm 



develops). Declarative knowledge is provided to the 
learner in a text form when errors occur or when the 
learner asks for help. For example, this is a typical 
message after making the Error of Omission (sec Table 
1 for error types) - Decrement (#1 and #2 are variables 
that are bound during rhe execution of the program) 
""You have to complete the borrow into the ones place 
by taking away one of the tens. So, the correct value for 
this area is Ul You typed #2 Please enter the correct 
value."* Although this is a very brief description of 
POSIT, it should suffice for the purposes of this paper. 
It is also necessary to provide a short description of the 
participants and the method used for the collection of 
data for this paper. 

Table 1. The four error categories and instances of 
eiTors in each category (numbers arc ranked according to 
number of occurrences). 

Errors of Errors of 

Omission Citmmission 

1. Increment (add 10) 5. /ncrcment 

12. Decrement (subtract 1) 4. Decrement 

13. Both 11. Both 

7. Neither 

Errors of Creation Other Errors 

8. smaller-from-larger 2. Fact error 
10.0-N=N 6. Typing error 
90-N=0 3. Leading 0 

14. N-N=N 

15. N-N=0 with regrouping 
from 

16. M-S=0, with S>M 
Jata Collectlc.n 

Twelve participants were used in the evaluauon. These 
participants were about 8 years old. Only those students 
who were having difficulties in subtraction were used. 
The selection of these participants was based on their 
teacher's recommendation and their performance on a 
subtraction test develope<l by the DEBUGGY system 
and reported in Van Lehn (1982). Out of a pool of 28 
students who were having difficulties in subu-action, 
twelve participants were selected, but only ten of these 
completed the study. The data was collected over a five 
week perifXl of time where the participants used the 
system about 30 minutes a day (averaging about 6 
hours of total insuiiction). POSIT saves a great deal of 
information about each interaction. These data were used 
to demonstrate the evaluation procedure dcscnbcd here. 
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Evaluation Procedura 

I begin with Litiman ar.d Soloway's notion of external 
evaluation. They suggest that CAI approaches to 
evaluation have been in terms of achievement testing. 
That is, groups who received instruction via CAI versus 
groups who did not are compared by using a pre and 
post achievement test The idea is that if the students in 
the CAI treatment out perform the students that did not, 
then the CAI is superior to no CAI. Littman and 
Soloway suggest that since ITS development is 
embedded in the field of cognitive science, a cognitive 
science perspective on evaluation would be more 
fruitful. 

By a cognitive science perspective, it is implied that the 
system attempts to understand the cognitive processes 
and/or the knowledge structures (states) of the learner. 
Therefore, the evaluation should follow these lines. 
That is, the evaluation should focus on how the student 
is performing some process or how the student has 
organized some knowledge rather than on a number 
correct as measured on a typical achievement test. A 
general achievement test does not get at the level of 
specificity that an ITS can examine. Hence, learning is 
viewed as a continuous process and the system would be 
evaluated in terms of its ability to recognize a learner's 
current state and how well the learner is helped to 
progress from the current state to an advanced state. 

However, gross analyses are also useful if they are well 
designed. I would say that much of the content of the 
ITS can be measured on a test that was dcvelqxxl for the 
evaluation of the content area of the ITS . The processes 
that learners are learning from the Hi can manifest 
themselves both in terms of the actual knowledge and 
processes, but also in the product of these processes 
(i.e., answers on an achievement test). Therefore, there 
should be a distinction made between these two levels 
of external evaluation. At one level, the ITS is 
evaluated in terms of the knowledge states and cognitive 
processes which are a part of the design of the ITS. At 
the other level, the ITS is evaluated based upon 
performance on an achievement test specific to the 
domain of the ITS. 

In terms of POSIT, the external evaluation Bf the first 
level would be in terms of the specific declarative 
knowledge (productions) that make up POi?IT. 
Operationally, this means that there should be ? 
reduction in the number of errors that POSIT identifies. 
That is, POSIT should be helping students eliminate 
errors in their algorithms. Table 2 shows the number of 
emrs on the first and last days of the field test made by 
the ten participants that used i^SIT. The first result of 
importance is the fact that there was an oven>U rcdi»ction 
in the number of errors (from 30 to 13 errors). 
However, on a cognitive science level, the results 
indicate that the specific error productions that the 
participants were using at the beginning of the field test 
were greatly reduced by the end of the field test. The 
most pronounced is the least significant - the leading 



zero error type. This result is not surprising since the 
regular classroom teacher taught the students to fill in 
all spaces, so leading zeros were correct in the 
classroom. On the other hand. Errors of Omission were 
nearly as frequent as the leading zero error. This decrease 
would indicate that the children's procedures vuerc being 
modified by their having used POSIT. Another approach 
might be to examine the number of steps that were 
carried out mentally and the time it takes between sieps 
(this data is available, but not analyzed). 

Table 2. The frequency of errors in each error category 
on the first and last days of the treatment. 



Type of Error 
Errors of Omission 
Increment (add 10) 
Decrement (subtract 1) 
Both 
Neither 

Errors of Commission 

Increment 

Decrement 

Both 

Errors of Creation 

smallcr-fmm-largcr 

0-N=N 

0-N=0 

N-N=N 

N-N=0 with regrouping from M- 
S=0. with S>M 

Other Errors 

Fact error 
Typing error 
Leading 0 
Inconect diag. 



First Last 



II 



I 
5 

n 
6 



2 
1 
t 



The second form of external evaluation v, carried out 
by using a pre and post test quasi experin jntal design. 
There was an overall decrease in the number of errors 
from the beginning of the experiment to the end (sec 
Orey, 1989). Perhaps a more powerful technique for 
conduciing this fonn of evaluation is to use the 
approach that Anderson, et al., (1985) used. That is, 
Anderson et al., used a procedure that was based on the 
notion of Bloom's (1984) "a-Sigma" Problem. The "2- 
Clgn^a** is the fact that the average student that is 
tutorea scores 2 standard deviations higher than the 
average student in a large group presentation 
instructional format. The "problem" is to find a cost 
effective solution that comes close to the effectiveness 
of human tutors. Anderson, el al., used three groups to 
evaluate their Lisp tutor. 

One group was tutored by competent human tutors 
(HT). One group was tutored by the computer tutor 
(CT). the third group was required to do their 
programming assignments on their own (NT, which is 
now assignments are typically complcro*!). The measure 
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was the time it took to complete the assignments. The 
HT group took the least amount of time (11.4 hours) 
then the CT group (15 hours) foltowed by the NT group 
(26.S hours). This type of evaluation seems compelling 
(POSIT is due to be evaluated by this method in the fall 
of 1990 and data for this approach should be available at 
the conference), at least for the administrator that is 
trying to make a decision about implementation. The 
former is more compelling for the researcher. Therefore, 
a good evaluation will use both forms of external 
evaluation. 

Internal evaluation, according to Littman and Soloway, 
is the evaluation of the relationship between the 
architecture and the behavior of the ITS. The point ef 
this evaluation is to maintain the goals of the tutor as it 
goes through its many revisions. A somewhat more 
powerful evaluation procedure was suggested by Priest 
and Young (1988) which might be used for internal 
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evaluation. This procedure revolves around a cognitive 
science evaluation procedure in that it evaluates the 
internal theoretical power of the learner model. 

Priest and Young (1988) suggest that there are four 
measures of the accuracy of the model of the learner. 
Tliose four measures are: 1) Percentage correct 
diagnosis; 2) Error fit measure; 3) Micro-theory 
evaluation quotient and, 4) The ciunulative hit curve. 
The first measure is the easiest, but there is no way to 
determine the true power of the learner model from this 
value. Internal evaluation, according to Littman and 
Soloway, is the evaluation of the relationship between 
th<: architecture and the behavior of the ITS. The point 
of this evaluation is to maintain the goals of the tutcM- 
as it goes through its many revisions. A somewhat 
more powerful evaluation procedure was suggested by 
Priest and Young (1988) which might be used fcM- 
internal evaluation. 



Figure 1. Priest and Young's (1988) four measures of internal theoretical power and their values for POSIT. 

1) Percentage cOTrcct diagnosis. 

Number of correct diagnoses X 100 = 371 X 100 = 76% for POSIT 
Number of data points 489 

2) Error fit measiu^. 

Number correct diag. - Number wrong predictions = 371 -46 = 0.665 for POSIT 
Number of data points 489 

3) Micro-theory evaluation quotient 

Ji = Number correct diag. - Number of productions = 371 - 16 = 0.726 for POSFT 
Number of data points 489 

4) The cumulative hit curve. 
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This procedure revolves around a cognitive science 
evaluation procedure in that it evaluates the internal 
ihcorctical power of the learner model. 

Priest and Young (1988) suggest that there are four 
measures of the accuracy of the model of the learner. 
Those four measures are: 1) Percentage correct 
diagnosis; 2) Error fit measure; 3) Micro-theory 
evaluation quotient; and, 4) The cumulative hit curve. 
The first measure is the easiest, but there is no way to 
determine the true power of the learner model from this 
value. 

AfteralU the system may have an unique explanatory 
production for every error made (Young and Priest refer 
to this as a trivial case, which theoretically it is, but 
pragmatically it would be mind boggling to develop 
such a system). This does not contribute to a theoretical 
view of learning and therefore is not a powerful theory, 
yet the percentage correct would be 100 percent. Notice 
that in Figure 1 POSIT has a percentage correct of 76 
percent 

The second measure, the error fit measure, is especially 
sensitive to false error analyses. Obviously, by 
subtracting wrong predictions, this value will decrease. 
The lower the number, the less viable the theory. 
Notice in Figure 1 that POSIT has a value of 0.665. 
The third measure specifically addresses the problem of 
a trivial theory. To ensure that the theory is robust 
enough, it is necessary to subtract the number of 
productions (or variant procedures) from the number of 
correct diagnoses. In this way, each production must be 
powerful enough to explain many errors or conditions. 
Perhaps the most powerful of the evaluation methods is 
the fourth: the cumulative hit curve. This curve plots 
the percentage of errors explained versus the number of 
productions used. The straight line represents the trivial 
case (1 to 1 correspondence between errors and 
productions). Figure 1 shows the cumulative hit curve 
for the first stage of evaluation of POSIT. 

These evaluation measures help evaluate the theoretical 
power of a learner model within the ITS. Further, we 
can judge our progress by showing how the new system 
out performs the old system. We can use an excepted 
guideline. However, it is not readily apparent whether 
POSIT is good or bad based on these numbers. What is 
needed is for all systems to report these numbers so that 
an industry standi can be established. 

In conclusion, no evaluation procedure is complete 
without some if not all of the procedures described in 



this paper. It is quite important to perform evaluations 
on an ITS both internally and externally. The external 
evaluation seems to be more important with regards 
Overall instructional effectiveness while the internal 
evaluation is concerned with theoretical power. ITS 
ai'fords the c^portunity to explore theoretical issues 
within cognitive science as well as instructional issues 
within the a cognitive instructional science. 
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HOW TO APPLY INTELLIGENT TUTORING DESIGN PRINCIPLES 
TO THE DESIGN OF COMPUTER-BASED INSTRUCTION 



Michael A. Orey, The Ufdversity of Georgia; and 
Wayne A. Nelson, Southern Illinois University at Edwardsville 



Abstract 

While many have suggested that a cognitive q>proach to 
the design of instruction is desirable* little change has 
occurred in our practice. On the other hand, researchers 
developing intelligent tutoring systems are using 
cognitive theory to design learning environments which 
actually extend our understanding of cognition. This 
paper describes how some of the cognitive design 
principles employed in intelligent tutoring systems 
might be utilized in the design of more traditional 
computer-based instruction. The end result is that the 
"riew" computer-based instruction will be closer in type 
to intelligent tutoring systems, where design decisions 
are made based on a cognitive perspective rather than the 
more traditional behavioral approach to design and 
development commonly found in current instructional 
systems design (LSD) models. 

The evolution of instructional systems has passed from 
teaching machines and programmed texts to computer- 
based instruction featuring tutorial and drill and practice 
strategies. Along the way, a systematic process for the 
design of instructional systems was also developed, 
prescribing various activities such as needs assessment, 
development of objectives, selection of strategies, and 
formative evaluation. Despite differences in outward 
appearances, all of these instructional systems have 
been driven by a frame-based approach derived from 
behavioral theories of human learning. For a variety of 
reasons, such systems and theories are now seen as 
largely ineffective for instruction. On the other hand, 
cognitive psychology has made great progress in 
describing the structure and processes of human 
cognition, and has fostered many discussions of the 
possibilities for improving instruction by relying on 
cognitive learning theories (Merrill, Li & Jones, 1990a; 
Hannafm & Rieber, 1989a; Hannafin & Riebcr, 1989b; 
Low, 1980; Salomon, 1985; Wildman, & Burton, 
1981; Winn, 1988). Glascr (1989) has noted that while 
there is still much descriptive theoretical work to be 
done, cognitive theories need to begin to address the idea 
of how those structures and processes are acquired. He 
feels that learning theory and instructional theory can be 
developed simi?ltaneously, stating that "[t]he study of 
learning can now take its cue from this knowledge 
[descriptive theories], and principled investigations of 
instruction can be a tool of major importance for the 
interactive growth of learning theory and its 
applicatk)ns" (p. 38). 

Current instructional design models are founded in 
behavioral psychology, and do not adequately support 
the kinds of analysis, design, development, and 



validation activities which are necessary for designing 
instruction which suiq)orts cognitive conceptions of 
learning. In order to design a new generation of 
computer-based instruction which incorporates the 
principles of cognitive learning theories, new models 
for the design of computer-based instruction are 
necessary to guide decision-making. In addition to 
fitting new ideas into old models, as has been suggested 
by Park, Perez, & Seidel (1987), and developing a 
completely new m.odel, as proposed by Merrill, Li, and 
Jones (1990b), we believe it is more feasible to borrow 
and adapt a model which has already proven effective. 
That model is the intelligent tutoring system. 

The evolution of intelligent tutoring systems has been 
based in cognitive science, reflecting the theoretical 
orientation of cognitive psychology better than much 
computer-based instruction which features page-turning 
and limited interaction. Various forms of computer- 
based instruction are incapable of performing well from 
a cognitive perspective because the design processes 
utilized are not conducive to the development of flexible 
instruction which can adapt to the learner as the 
instruction occurs. As Winn (1988) has noted, 
traditional instructional design models require that all 
decisions about instruction be made and tested before the 
instruction is implemented. Winn (1989) suggests, 
therefore, that intelligent tutoring systems may be the 
instructional medium of the future because of the 
dynamic cognitive modeling capabilities which allows 
the system to make adaptive instructional decisions as 
the learner uses the system. In the following pages, a 
general model for an intelligent tutoring system is 
described, along with suggestions for the iniprovement 
of computer-based instruction using design principles 
derived from research in cognitive science. 

A Model of an Intelligent Tutoring System 

Intelligent tutoring systems derive historically from 
computer-based instruction, but since there are basic 
differences in theoretical perspectives, they represent 
two poles along the spectrum of systons. Differences 
between the two types of instructional systems are 
primarily structural characteristics: intelligent tutoring 
systems encode knowledge, while computer-based 
instruction encodes instructional decisions based on 
knowledge (Wengcr, 1987). Subject matter is separated 
from teaching method in an intelligent tutoring system, 
thereby separating content from presentation techniques 
(Clancey, 1984). Intelligent tutoring systems are based 
on the idea that natural learning occurs through context- 
based performance, with the goal of research being to 
develop programs which understand student 
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misconceptions and behave as a master teacher would to 
provide appropriate instruction (Loser & Kurtz, 1989). 
Because of the architecture of the software, intelligent 
tutoring systems allow comparisons between learner 
behavior and an expert model in order to provide the 
appropriate instructional activities, just as a teacher 
would 

The main advantage of the intelligent tutoring systems 
model, however, is that the underlying assumptions 
about learning and instruction differ from those of 
current instructional design models. Regardless of the 
particular model (c.f. Hartley & Sleeman, 1973; Hayes- 
Roth & Thomdyke, 1985; Park & Seidel, 1989; 
Wenger, 1987), all intelligent tutoring systems 
incorporate at least four major components: 1) the user 
interface; 2) the learner model; 3) the expert model, and; 
4) pedagogic knowledge. As depicted in Figure 2, the 
learner and expert models are often combined into an 
area where diagnosis occurs through comparison 
between correct knowledge stales (the expert model) and 
incorrect knowledge states (the learner model). 

The interface is how the system communicates with the 
learner. This is a two-way communication where the 
learner is engaged in some activity while the system \s 
interpreting the learner's activity so that a meaningful 
response can be made. The expert model is typically a 
database of correct knowledge states for a given domain. 
These states may be in the form of declarative or 
procedural knowledge organized in such a manner that 
the knowledge of an expert in the domain is accurately 
represented. Expert knowledge may be supplied to the 
system, generated dynamically, or both. The learner 
model is a representation of the errors or 
misconceptions that commonly occur when learners are 
exposed to the content, but can also include other 
information, such as a curriculum map of the learner's 
sequence through the curriculum. The system attempts 
to ascertain the knowledge that the learner possesses in 
order to be performing at the level and manner currently 
being exhibited. This diagnostic process is 
accomplished by comparing the learner's present 
knowledge state with knowledge in the expert model. 
The results of the comparison are then passed to the 
pedagogical model, where decisions are made about 
what, when, and how information is to be 
communicated to the learner. 

Design Principles for 
Computer-Based instruction 

It is our contention that some aspects of each of the 
components in Figure 2 can be included in any type of 
computer-based instruction. The interface is important 
regardless of the type of software being developed, and 
in order to create instructional software, the content 
needs to be clearly defined and represented (the expert 
model). The learner is always considered, regardless of 
the delivery system, and instructional software must, by 
definition, contain knowledge about how to provide 
instruction (the pedagogical model). Therefore, the 



design and development of computer-based instruction 
and intelligent tutoring systems are not vastly different. 
The following discussion, while not intended to be an 
exhaustive review of the literature, presents general 
guidelines for designers of computer-based instruction 
derived from research in cognitive science arKl intelligent 
tutoring systems. 
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Figure 2. Components of an Intelligent Tutoring 
System. 

The Interface. Much significant research has been 
completed in the area of hunum-computer interaction, 
and many principles based on cognitive theories have 
been proposed for user interface design (c.f. Card, 
Moran, & Newell, 1983; Norman & Draper, 1986; 
Shneiderman, 1987). The xaz]cx problem is one of 
communication with the learner, since the 
communication is delayed. The designer must anticipate 
the possible actions a user may take, and devise ways to 
handle the actions bdatt the user ever interacts with the 
software. This requires attention to aspects of learning, 
performance, subjective satisfaction, and retention of 
information as related to the interface, supported by 
iterative development, testing, and revision of the 
interface components (Shneiderman, 1987). 

One of the primary considerations in designing a user 
interface should be ease of use. It is important to 
remember that the learner is not only learning the 
content, but is also learning how to operate the 
software. Screen design, use of special function keys, 
menu selection, and feedback on enors are just some of 
the aspects which need to be considered (c. f. Jay, 
1983). The key in designing these components is 
consistency, so that the learners can make use of their 
knowledge about operating the software as they move 
from one screen to another, or from one program to 
another. Consistent interface design will also help to 
reduce the cognitiv-; load on the learner (Norman & 
Draper, 1986; Shneiderman, 1987). For example, 
Anderson and his colleagues (Anderson, Boyle, & 
Reiser, 1985; Anderson & Reiser, 1985) have designed 
an interface to reduce the amount of information abou: 
the software that the learner must keep in memory, 
thereby minimizing interference between the learning 
task and operation of the software. 

Interaction style is also an important part of interface 
design. An interaction style which is consistent with 
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the learners knowledge of computers is most desirable. 
There are many possibilities, including command- 
driven, menu selection, and direct manipulation 
interfaces. Depending on the leamer*s expertise, one 
style might be more appropriate than another. For 
example, if the learner is a skilled computer user, a 
command-driven interaction style may be the most 
appropriate, but if the learner is relatively inexperienced, 
direct manipulation of objects with a mouse may be 
more desirable. 

Metaphor can also be a powerful device for an effective 
inter&ce, providing students with "comparisons which 
can help them learn" (Carroll & Mack, 1985; p. 40). 
Much of the applications software currently being 
marketed makes use of metaphor to help users be more 
productive. We have windows, trash cans, desktops, 
buttons, cards, folders, and pages. Such metaphors 
enable learners to build on their past experiences, and to 
map concepts in a well known domain to similar 
concepts and relations in a new domain. Complex 
metaphors may require sophisticated graphics 
capabilities and direct manipulation, but the beneHts to 
the learners are worth the efforts. There arc many 
metaphors in the classroom which are familiar to 
students, and which could be incorporated into an 
effective interface for learning. 

The interface is not cmly a means for input and G\itpui 
of information, but can also supply important data 
about the learners. T)cpcndmg on the domain, data from 
the interface can be used to monitor the learning process 
as it unfolds. If the content is process-oriented, the 
interface should be designed to monitor the learner's 
progress. For example, Orey and Burton (1990) 
designed an interface which supplied the system with 
more information about the child's subtnu:tion process 
than just the answer to a problem. These data flowed 
the system to make decisions about the nature of the 
learner's subtraction errors. If the content is declarative, 
the interface should facilitate retention and organization. 
For example, the way in which an expert organizes 
knowledge in the Held of her expertise might be depicted 
in some graphical form such as a content map, with the 
intention that the learner will develop a similar 
organization in obtaining domain proficiency. 

The Expert Model, the Learner Model and 
Diagnosis, Experts in a domain organize knowledge 
and control problem-solving processes differently than 
novices (Glascr, 1989). The goal of instruction from a 
cognitive perspective, then, is to replicate the 
knowledge structures and processes of the expert in the 
mind of the learner (Wildman & Burton, 1981). In order 
to do so, the domain expert's knowledge must be 
mapped into symbols a computer can store and 
manipulate, and be presented to the learner in an 
organized manner. The appropriate form for knowledge 
representation is determined, in part, by the kinds of 
diagnostic procedures being implemented. Van Lehn 
(1988) describes the notion of bandwidth, which is the 
correspondence between observable actions of the learner 
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and mental states within the ieamer. High bandwidth is 
the ideal case of a one-to-one correspondence between 
the learner's mental states and observable actions. Low 
bandwidth, on the other hand, would be the case when 
there are multiple mental states between observable 
actions performed by the learner. Where the system fits 
on the continuum between high bandwidth and low 
bandwidth is largely determined by the interface. For 
example, Orey and Burton (1990) used a process 
oriented interface to access as many actions as possible 
as the learner performed whole number subtraction. On 
the other hand. Brown and Burton (1978) used a system 
that only examined the completed answers to an entire 
subtraction test There are a multitude of mental states 
that occurred before the system was able to diagnose 
errors. The result of the bandwidth diagnostic level for 
these two systems was that the diagnostic strategies and 
knowledge representation were entirely different 

The designer of computer-based instruction can make 
use of knowledge representation and diagnostic 
principles by focusing on high bandwidth diagnosis. 
First, an interface to support this level of diagnosis 
needs to be designed Second, the most conunon types 
of errors that may occur need to be identified in order to 
construct a partial "modeP of the learner that can be 
used for diagnosis. The type of error made by the learner 
can then be determined with extended conditional 
statements that are checked at the point of the error 
(context-specific error checking). In this way, a "mini" 
high bandwidth diagnostic system which uses an 
approximation of a production system can be 
implemented. 

Simulation software is another area of computer-based 
instruction where the monitoring and diagnosis of user 
errors can be effectively applied For example, suppose 
the simulation involves the operation of a nuclear 
power plant The instantiation of a major error might 
result in the melt down of the simulated power plant, 
which may have dramatic effects on the learner's 
memory of the error. The error which caused the melt 
down, if properly communicated to the user, would 
hopefully not occur in the real world environment of 
running the power plant 

The Pedagogical Model, The pedagogical model 
contains knowledge necessary for making decisions 
about the teaching actions to be taken by the system. 
Since there tends to be considerable overly between the 
functions of the various components of an intelligent 
tutoring system, the decisions and actions of the 
pedagogical model arc highly dependent on the results of 
the diagnostic process. In general, the pedagogical 
model must decide when to present information to the 
learner, how to present the information, and what 
information to present. There have been a variety of 
pedagogical approaches employed in intelligent tutoring 
systems, but most of the current systems tend to 
implement only one pedagogical strategy. Intelligent 
tutoring systems, therefore, do not yet possess a rich 
repcjtoire of tutoring strategies from which to select, in 
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pan because little is known about how to teach, 
especially in a one-to-one setting common in tutoring 
(Ohisson. 1987). 

In general, pedagogical strategies are dependent on the 
overall context of the learning environment embodied in 
the intelligent tutoring system. Three general types of 
environments have been implemented: systems which 
monitor student activity within a problem-solving 
domain, systems which employ mixed-initiative 
dialogue between learner and tutor, and systems which 
en^nloy guided discovery or coaching (Wenger, 1987). 
The choice of tutoring environment is dictated by the 
nature of the content to be learned, the knowledge and 
experience of the learner, and the assumptions about 
learning inherent in the underlying theory on which the 
system is based. Current research with intelligent 
tutoring systems primarily focuses on context-based 
environments where students learn bv working on 
specific, real-world problems, because cc jnitive theories 
have shown that knowledge and expertise is acquired by 
active application of knowledge during problem solving 
(Glaser, 1989). Further, learning strategies which make 
use of modeling of specific and appropriate procedures 
and strategies for problem-solving, and which 
strengthen existing knowledge through practice which 
minimizes error, have been successfully applied in 
intelligent tutoring systems research (Anderson, Boyle, 
& Reiser, 1985; White & Fredricksen, 1990). While 
these strategies are not significantly different from those 
already employed in some computer-based instruction, 
the use of context-based strategies which support 
situated cognition (Brown, Cbllins, & Duguid, 1989) is 
becoming an important factor in the development of 
intelligent tutoring systems which is not emphasized in 
current computer-based instruction, especially drill and 
practice and tutorial software. 

Within a particular learning environment, a variety of 
teaching tactics can be implemented. Some of the 
possible tactics are presented in Table 1. Interested 
readers are referred to Wenger (1987) and Ohisson (1987) 
for more detailed discussions of teaching tactics 
employed in intelligent tutoring systems. The selection 
and use of teaching tactics are governed by the 
characteristics of the learning environment, along with 
the general pedagogical approach. Various teaching 
tactics can be selected using an opportunistic approach 
(results of diagnosis reveal when opportunities for 
intervention arc appropriate), or a plan-based approach 
(the focus of diagnosis is to monitor teaching plans and 
goals). Of course, a human tutor probably uses some 
combination of opportunistic and plan-based 
interventions, and research is currently focusing on the 
development of more sophisticated pedagogical 
techniques combining aspects of both pedagogical 
approaches (Wenger, 1987). 
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Table 1. Some teaching tactics employed in 
intelligent tutoring systems. 



Presenting 
Information 


Monitoring 
Performance 


Error 
Detection 


Socratic 
dialogue 


Guided practice 


Revealing errors 
on occurrence 


Demonstration 


Annotated 
practice 


Marking for 
later explanation 


Priming 


Hints 


Probability 
thresholds 


Associative 
Links 


Prompt self- 
review 


Bug repair 


Curriculum 
Maps 


Prompt self- 
annotation 




Issues and 
Examples 


Evaluate 
hypotheses 




Answ^ 
Questions 


Coaching 






Interactive 
simulation 
(Microworid) 





Many opportunities for integrating pedagogical 
strategies derived from research on intelligent tutoring 
Sjotems are available for designers of computer-based 
instruction. For example, the work on conceptual 
models, as discussed by Mayer (1989), can provide an 
effective frameworic for applying cognitive theories of 
learning in computer-based instruction systems. As an 
illustration of this approach, consider the study 
conducted by Wertheimer (1959), which examined the 
differences between teaching children the formulae for 
the area of different geometrical shapes versus teaching 
children the conceptual explanation of those same 
formulae (see Figure 3). Initially the learner is given a 
paper parallelogram and a pair of scissors (Step 1 of 
Figure 3). 

The child is then instructed to cut off the triangle on 
one side of the parallelogram (Step 2 and 3) and place it 
on the otl»er side (Step 4). From this experience, the 
learner understands that the formula for the area of a 
parallelogram is A=B*H (Where A is the area, B is the 
base and H is tlie height, or in Figure 3, A=7«3=21). 
Direct manipulation or animation could be used to 
implement this strategy in computer-based instruction 
(Orey, 1985). The learner could be coached through the 
process, learning to "grab" the triangle with a mouse 
and place it on the other side of the parallelogram. Such 
an approach puts the learner in more control of the 
environment, and is consistent with the pedagogical 
strategies employed in intelligent tutoring systems. 

Pedagogical knowledge, then, can be as important in 
computer-based instruction as it is in an intelligent 
tutoring system. There are many decisions to be made 
regarding not only strategies, but sequencing, and media 
(graphics, text, sound, or video). Certainly, more 
traditional approaches to computer-based instruction can 
be used as components of the pedagogical knowledge 
base (see Jonassen, 1988). Meiacognitive considerations 



may also be incorporated within the pedagogical 
knowledge of a computer-based instructional system. 
That is, communication with the learner can be used to 
induce reflection on the learning process or to suggest 
alternative problem-solving Strategies (Brown. 1975). In 
whatever manner, it is important for the designer to 
remember that learners are capable of controlling their 
own learning to some extent (Linn & Clancey, 1990; 
Winne, 1989). 




3 4 

Figure 3. Wertheimer's (1959) conceptual model for 
teaching area. 

Conclusions 

We have sketched an approach to the design of 
computer-based instruction derived from cognitive 
psychology, in particular tlie design principles used in 
intelligent tutoring systems. As the analysis unfolded, 
we were struck by the idea that there were many 
similarities between intelligent tutoring systems and 
computer-based instruction that has been devd(^)ed from 
a cognitive perspective. That is to say. rather than a 
dichotomy existing where the two poles are computer- 
based instruction and intelligent tutoring systems, 
actually there is a continuum anchored at one end by 
traditi(xial computer-based instruction developed from an 
instructional systems design perspective (such as that 
found in many training settings), and at the other end by 
the "ideal** intelligent tutoring system. Some intelligent 
tutoring systems are not quite the ideal, and some 
computer-based instruction has been developed from a 
cognitive perspective. These systems overlap in the 
middle of the continuum. However, there is not a great 
amount of computer-based instruction developed from a 
cognitive perspective. As a discipline, we need to begin 
to shift toward the intelligent tutoring system. If we are 
to grow intellectually as a field of inquiry based in 
learning tlieory. then we need to be not only embracing 
the principles of cognitive psychology/science (which is 
a growing learning theory rather than a stagnate one), 
but applying it to instructional design and testing it in 
real learning settings. 
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THEORETICAL FOUNDATIONS OF INSTRUCTIONAL APPLICATIONS OF COMPUTER- 
GENERATED ANIMATED VISUALS 



Lloyd P. Rieber and Asit S. Kim, Texas A&M University 



Abstract 

Graphic tools abound in the development of coniputer- 
based instruction (CBI). This, coupled with the tendency 
to incorporate features particular to tlie computer 
medium has resulted in a proliferation of graphics in 
CBL Animation is frequently incorporated into CBI, 
though designers and developers rarely have a firm 
rationale for its use. This paper discusses theoretical 
perspectives relevant to the application of animation in 
CBL Perceptual theories are presented which help to 
explain how the illusion of motion is actually induced. 
Dual-coding theory is presented to explain how visual 
information is learned and remembered. This 
information is intended to familiarize designers and 
developers with the theoretical underpinnings which 
explain and support the use of animation in CBI. An 
understanding of this information should help guide 
decision-making of instructional applications of 
animation. 

Introduction 

Computer graphics, when designed appropriately, aie 
believed to enhance learning from computer-based 
instruction (CBI) (Alesandrini, 1987). Several 
advantages of graphics are frequently noted. They can aid 
visualization of complex relationships between concepts 
and rules in short-term memory (Cooper & Shepherd, 
1973). They can act as powerful mnemonics for 
remembering verbal information and concrete concepts 
(Paivio & Cs^, 1973; Pressley, 1976). Additionally, 
graphics are commonly believed to be effective 
attention-gaining devices and help increase the affective 
appeal of learning tasks and materials, though research 
on these issues is scant at best (eg. Surber & Leeder, 
1988). 

One graphic feature which has been excessively popular 
among CBI designers is animation. Computer 
animation can be defmed as a series of rapidly changing 
computer screen displays that present the illusion of 
movement (Caraballo, 1985). Animation, whether on 
computer, film, or otherwise, is not real motion, but 
only its representation. Even the most sophisticated 
computer graphics systems operate on a "draw, erase, 
change position, draw" repetition to produce the illusion 
of motion. However, the theoretical and empirical 
foundations to explain and support the use of animation 
in ;nstruction have not been firmly established. 

Resitarch on the use of instructional visuals, of which 
animation is a subset, has a long history. While visuals 
are constantly prone to distraction effects (Willows, 
1978), conditions under which graphics can enhance 



learning have been described. The most notable of 
which is congruency between the learning task and 
instructional activities (Levie & Lentz, 1982; Levin, 
Anglin, & Carney, 1987; Levin & Lcsgold, 1978). 
Instructional research on animation is much more 
recent, due, in part, to widespread availability of 
graphics tools in CBI authoring in recent years. 

Animation research has focused on two fundamental 
applications of animation in CBI: 1) animation as a 
presentation strategy; and 2) practice activities 
involving interactive animated graphics (Rieber, 1990a). 
Recent reports have offered encouraging results for both 
^plications (Reed, 1985; Rieber, 1990b, 1990; Rieber, 
Boyce, & Assad, 1990; White, 1984). Efficacy of 
animated presentations appears particuL^dy dependent on 
task requirements, cognitive load, and attention-gaining 
devices. 

The purpose of this paper is to describe several 
perceptual and psychological frameworks which support 
instructional applications of animation (see Rieber, 
1990-a for a review of animation research conducted to 
date). First, perceptual factors are described which help 
to explain how the illusion of animation is made 
possible. This information should help guide designers 
to construct more convincing animated displays. 
Second, current views on how visuals contribute to 
long-term retention are discussed in the context of 
instructional design. These theoretical perspectives 
directly impact on why, when, and how to incorporate 
animated visuals effectively into CBL 

Perceptual Factors 

Visual cognition is the process of how people perceive 
and remember visual information (Pinker, 1984). 
Perception is the process of selectively attending to and 
scanning a given stimulus, interpreting significant 
details or cues, and fmally perceiving some general 
meaning (Levie, 1987). Students perceive meaning from 
animated visuals when they are tricked into seeing 
something that really is not there. Certain perceptual 
factors help to explain this phenomenon. 

Animation is an example of apparent motion which is 
the phenomenon of perceiving motion when there is, in 
fact, no physical movement of an object in the visual 
field (Ramachandran & Anstis, 1986). (In contrast, real 
motion is experienced when an object actually moves 
across the receptive field of visual detecting neurons. 
The object crosses receptive fields of at least two 
neurons so that movement is perceived [Schoutcn, 
1967].) Motion is perceived for two or more spatially 
separated and static objects which are alternately 
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presented to the observer over time. Consequently, there 
is no actual motion of the image on the eye's retina. 
The visual system perceives motion by taking and 
combining discrete information into a smooth and 
continuous set. In other words, the mind, when 
confronted with a r^id series of still images, fills the 
gaps between the frames resulting in the perception of 
continuous motion. An everyday example is the popular 
use of neon lights io create a dynamic storefront 
display, such as moving arrows, to attract customers' 
attention to the shop. 

The most intensively studied version of apparent 
motion is that perceived when two lights are presented 
at different times and different locations. This produces 
stroboscopic motion (also called Beta movement) 
(Kaufman, 1974; Schiffman, 1976). When images are 
shown at a rate of less than 16 frames per second, the 
motion appears choppy or jumpy. When individual, 
non-moving images are shown collectively at around 16 
frames per second, the motion is perceived as smooth 
and continuous. Even the most inexpensive computer 
systems available today easily match this criterion. 

Closely related to stroboscopic motion is the phi 
phenomenon (Schiffman. 1976). This is the illusion of 
motion produced when stationary lights are turned on 
and off in sequence. Examples of this phenomenon 
include theater marquee lights and scoreboards in sports 
arenas. The phi phenomenon also accounts for 
animation produced by the coordinated switching of 
pixels on computer displays. Both stroboscopic motion 
and the phi phenomenon are explained by several 
perceptual theories. (Other examples of apparent motion 
have been identified, such as the delta phenomenon, 
which is produced by varying the brightness of the 
stimuli [William, 1981].) 

Of the many factors that determine the nature of 
apparent motion, three are particularly relevant in the 
design of animation on computer displays — ight 
intensity of the displays, time between projection of the 
separate displays, and the spatial distance between each 
of the displays. For example, two lights will be 
perceived separately (i.e. no perceived motion) when 
they are alternately presented at too slow a rate. 
Conversely, if they are alternated at too fast a rate, then 
the two points are perceived simultaneously, again, 
precluding the perception of motion. Beyond the 
individual importance of these factors, Korte*s Law 
holds that apparent motion can only result when these 
faaors are synchronised properly (Korte, 1915, as cited 
in Kaufman, 1974, and Carterctte & Friedman, 1975). 
When one factor is held constant, the other two factors 
vary proportionally (though not necessarily directly). 
For example, if light intensity is held constant, the 
spatial distance between the displays for optimal 
apparent movement varies with the time interval 
between the displays. So, as the displays are moved 
further apart, the time between the projection of the 
displays must also increase. Other factors, such as 
spatial ori^tation, spatial frequency, depth plane, color. 



size, shape, and texture of the displays, also impact the 
perception of apparent motion, but the influence of each 
is considered quite small. 

Psychological research has only now been able to 
describe the mechanisms by which the visual system 
(i.e. retina and brain) perceives apparent motion. In 
order to perceive an intermittently visible object as 
being in continuous motion the visual system must 
detect correspoi ience between the sq)arate displays of a 
single object in motion (Mack, Klein, Hill, & 
Palumbo, 1989; Ramachandran & Anstis« 1986; 
Ullman, 1979). This is a top-down mental process in 
which the brain applies one organized and meaningful 
pattern to the separate i^nages. For example, an 
animated scene of a person running across the computer 
sc*'een may actually consist of 30 sq)arate frames. The 
visual system '^sees'* all thirty fmnts but perceives only 
one object in motion. Detection of correspondence 
results in a type of cognitive ''assembling** of the 
separate images into one meaningful set Motion acts as 
the **glue** in this assembly process. If each successive 
display differs slightly from the previous one* 
correspondence detection is achieved and the visual 
system perceives motion in a single object Explaining 
how the visual system solves the problem of 
correspondence detection is controversial. Several 
explanations have been offered. Over the years, 
perceptual psychologists have shifted away from the 
view that apparent motion is the result of a single 
psychological process to a two process iheary. This 
latter view holds that there are two distinctly qualitative 
processes at woric long-range and short-range apparent 
motion (Braddick, 1974« as cited in Petersik, 1989; 
Julesz, 1971). Research indicates that motion is 
perceived using either or both of these processcs(Therc 
is competition between the two processes when each is 
equally stimulated The conditions describing when one 
or both processes are elicited have not, as yet, been 
adequately determined [Pantle & Picciano, 1976].) 

In the short-range process of perceiving motion, a 
display first stimulates the retina and the image is 
converted into an array of tiny points of varying 
brightness. This image is converted into an electrical 
signal which is carried along the optic nerve to the 
brain. The brain then compares each point with 
corresponding points in each succeeding display and 
discerns that one set of matched points composing a 
single object has changed its position by means of a 
yet-to-be-understood complex mental computation 
(Petersik 1989). The brain then conducts a point-to- 
point matching by invalidating hundreds of potentid 
matches and deeming them to be false. 

The long-range process of perceiving motion provides a 
different explanation by contending that the visual 
system applies strategics that limit the number of 
matches that the brain needs to consider and thus avoids 
the need for a complex point-to-point comparison 
(Ramachandran & Anstis, 1986). This view suggests 
that the visual system uses a numbci of tricks evolved 
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over the course of thousands of years. By taking 
advantage of the regularities of the physical work, the 
visual system extracts salient features from a complex 
display such as clusters of dots rather than individual 
dots and then searches for those features in the 
successive images. The features could be edges, short 
outlines, or areas of low spatial frequencies (blotches of 
brightness or darkness). Texture is another feature that 
appears salient to the visual system. 

In the face of ambiguity, the visual system limits its 
perception of motion to those consistent with universal 
physical laws. It assumes that the physical world is not 
chaotic and random but organized and predictable. By 
capitalizing on these predictable physical properties, the 
visual system perceives linear motion in "preference" to 
one of abrupt change. The visual system also assumes 
that a moving object will progressively cover and 
uncover portions of a background. This means that if an 
opaque object temporarily covers the background, the 
unseen background is still perceived to exist 
(Ramachandran & Anstis, 1986). 

In summary, apparent motion forms the basis of 
motion pictures (including films, videotapes, and laser 
discs), theater marquees, scoreboards and animation in 
computer software. In each case, the rate of picture 
presentation is crucial for the motion to perceived as 
smooth and continuous. Apparent motion occurs when 
a visual image persists for a brief period of time even 
though the physical stimulus ends. When photographs 
m shown at a proper rate, the image from each is taken 
and fused with proceeding and succeeding ones to form a 
perception of steady movement Successive discrete 
images are integrated to produce an apparently 
continuous visual environment 

Long-term Memory structures and 
Some Instructional Applications 

Humans appear to be particularly adept visual learners 
(Kobayashi, 1986). The predominant theoretical 
explanation for this is dual-coding theory (Paivio, 1979, 
1986). This theory suggests that there are independent 
encoding mechanisms, one visual and one verbal. 
Although competing hypotheses exist, such as those 
suggesting that all information (visual and verbal) is 
stored in prepositional forms (Pylyshyn, 1981), dual- 
coding theory has considerable empirical support 
(Anderson, 1978). 

Dual-coding theory contends that pictures and words 
activate independent visual and verbal codes and makes 
several assumptions. The first is that separate coding 
mechanisms are additive, so that something coded in 
both picture and verbal forms is more likely to be 
remembered (Kobayashi, 1986). The second assumption 
is that the availability of these two coding mechanisms 
differ in that a picture is more likely to be coded 
verbally and pictorially than words (Hannafin, 1983). 
Pictures are often remembered better than words because 
they are more likely to be encoded redundantly than 



words. The probability of recall is increased due to the 
availability of two mental rq)resentations instead of just 
one. If one memory trace is lost (whether visual or 
verbal), the other is still available. 

Dual-coding is more likely to occur when the content is 
highly imageable (Paivio & Csapo, 1973). Concrete 
concepts, like "tree" or "chair", are good examples of 
information which readily produce internal images in 
most people. Conversely, people do not automatically 
form internal images for abstract concepts, like "justice" 
or "freedom". In these cases, it is often useful to provide 
the learner with a prototype image which communicates 
the most sal^^nt features of the concept. This 
prototypical image is frequently analogical to the 
concept, like a blindfolded person holding a set of scales 
to represent justice. Most of the supportive research 
shows that words, sentences, and paragraphs that are 
highly imageable are recalled better than those which are 
not and thai the learning of concrete concepts precedes 
the learning of abstract concq)ts^oncrete concepts are 
stored as images whereas abstract concepts are stored as 
verbal representations (Paivio, 1986). 

Animation, like any picture, should aid recall when it 
illustrates highly imageable facts, concepts, or 
principles. Differences between animated graphics and 
static graphics is not as clear, but animated graphics are 
probably better at communicating ideas which involve 
changes over time because of its ability to represent 
motion, thereby reducing the level of abstraction of 
temporal ideas. For example, the motion of an animated 
car traveling from Boston to Washington accompanied 
by a display of the miles traveled and gallons of 
gasoline consumed helps to concretize the mathematical 
rule for computing average "miles per gallon". Static 
visuals represent, at best, visual "snapshots** of such 
ideas and are often accompanied by abstract symbols, 
such as arrows and dotted lines, in the hope of 
suggesting the motion attribute in static form. Learners 
must consciously work to connect and integrate these 
snapshots into a meaningful display, whereas animation 
triggers the automatic ability of the visual system to 
induce apparent motion, thus freeing short-term 
memory for other tasks. 

Instruction often involves concepts and rules which not 
only change over time, but also in a certain direction. 
The direction of the path of travel of aii object is defined 
as its trajectory. For example, many ideas in physical 
science, such as classical mechanics, not only require 
learners to understand that an object is moving at a 
certain speed, but also that it is moving in a certain 
direction. A case in point is velocity, which is defined 
as the speed and direction of a moving object. 

If a learning task only requires learners to visualize 
information, then the use of static visuals would be 
sufficient. However, if the task demands that learners 
understand that an idea changes over time or involves 
directional characteristics, then static visuals, at best, 
can only hope to prompt learners to menially construct 
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motion and frajcciory on their own (Often, merely 
prompting learners to iniemally image can be sufficient 
to produce learning effects. Studies involving static 
visuals have indicated that adults are much more likely 
to spontaneotisly form internal images than children, 
suggesting maturation effects [sec Pressley, 1977].) 
However, animation makes this cognitive task more 
concrete by providing motion and trajectory attributes 
directly to the learner, thus allowing a more prcisc 
image to be perceived and potentially stored in long 
term memory. 

Conclusion 

Due to the wide scale and increasing availability of 
computers as an instructional delivery system and recent 
popularity of graphic tools, instructional applications of 
animation are destined to increase. In addition, the rapid 
increase in graphical user interfaces (GUI) on a wide 
range of computer platforms is unprecedented and this 
trend is also likely to increase. The purpose of this 
paper has been to provide instructional designers with 
background information related to the application of 
animation in CBI. Two main ideas have been presented. 
First, several perceptual theories were presented to 
describe how the illusion of animation is made possible 
on computer displays. Second, a dual-coding model was 
pru.ntcd which provides general theoretical support for 
the storage and retrieval of pictorial information in 
long-term memory. Dual-coding is widely accepted as a 
suitable model for predicting picture facilitation effects 
(although it is not meant to represent a physiological 
model) (Anderson, 1978). 

The fact that computers are capable of creating this 
illusion should come as no great surprise to most CBI 
developers as most commercial software packages 
already incorporate a myriad of examples of animation. 
However, an understanding of the underlying perceptual 
mechanisms should help in the design of more effective 
and convincing computer displays. It should also make 
us more appreciative of the human visual perception 
system. Motion perception is attributable to human 
capabilities much more than computer technology. An 
individual's schema, or organized network of world 
knowledge (Norman, 1982), has evolved to accept, 
expect and interpret motion. An individual ^'sees** 
motion in a collection of carefully coordinated static 
lights through top-down mental processing. This ability 
has evolved as a result of living in a dynamic world 
where very little information is truly static (Goldstein, 
Chance, Hoisington, & Buescher, 1982). Therefore, 
animation can help to represent the natural world in 
contrived settings, such as computer displays. 

From an instructional point of view, this means that 
many ideas can be presented ai.d understood in a natural 
and concrete way on the condition that motion and/or 
trajectory are important attributes to the ideas being 
conveyed. Riebcr (1990a) has described four basic 
applications of animation in instruction: cosmetic, 
attention-gaining, presentation, and practice. Each 



application demands different development 
considerations. Of course, the fact that animation is 
incorporated into software in no way speaks to the 
effectiveness of the displays when the intent is to teach 
one or morc intellectual skills. Instructional visual 
research has provided designers with one global 
principle: there arc times when visuals can faciliute 
learning, times when visuals do not aid learning but do 
no harm, and times when visuals distract. Having the 
capability to produce animated displays does not 
automatically grant the designer the wisdom to use 
these techniques appropriately. 
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THE INTERPLAY OF INTERACTIVITY AND MOTIVATION IN EDUCATIONAL SOFTWARE 



Daniel L. Schwartz and Jeanne Buckley, Teachers College, Columbia University 



Abstract 

Invcsiigaiing the relationship between a program's 
interactive format and its motivational strengths, it was 
found that Sth grade students categorized educational 
software according to interactive genre. Results of a 
motivational ranking and clinical interviews indicate 
that each interactive genre requires an appropriate 
configuration of motivational elements to be app^ing. 
It is recommended that designers capitalize on children's 
prior conceptions of interactivity to increase the appeal 
of educational software. 

Introduction 

This work explores the interplay between various types 
of software interactivity and the elements of a program 
that are intended to motivate the child. It is our 
hypothesis that children bring specifiable expectations 
to their interactions with educational software. These 
expectations, to a large extent, will determine which 
features of a given program will be motivating to a 
child. Additionally, the different genres of interactivity 
relied upon by sofrvare designers will elicit different 
expectations from a child and may resonate or conflicl 
with the motivating elements built into the program. 
The state of affairs for the designer of educational 
software is much like that for the movie director. The 
director of a ceitain genre must include the elements the 
audience :ome$ to see (e.g., the scary scenes of a horror 
movie) and not fully replace or confuse them with 
elements that work for other genres (e.g., slapstick). 
Similarly, the software designer must be attentive to the 
connections between the interactive elements upon 
which sde relies and those elements that are appealing 
to a child. 



across software. For example, Malone (i981) writes of 
the role of challenge in creating motivation. Challenge, 
however, is not what motivates us to use a particular 

word processing program.^ Rather we choose those 
programs that conform to our needs as writers. We 
intend to show that the motivating elements of software 
vary from program to program and are dq)endent to a 
large degree on the interactive premise of the software. 

To begin exploring the relationship of interactive 
software and motivation we coltected three sets of data: 

• By having fifth graders sort software titles into 
piles of "like" programs, we investigated how 
chikiren cat^orize educatxmal softwsoe. 

• We gathered information about the 
motivational power of programs by asking the 
students to indicate on a scale of one to three 
how much they like to use each program. 

• Through group interviews we identify the 
interactive elements of the favored software 
genre *advemmes. 

All of the studies were conducted with fifth graders 
enrolled at Royale Elementary School in Darien, 
Connecticut The Darien School Distria has an ample 
computer program that is labcHBtory based Children use 
the computer lab during &ee time and regularly as part 
of class time. The software selection and usage is 
closely linked with the curriculum. There are enough 
computers for children to work individually if they 
choose. And, a full 75% of the children had computers 
(or video games) in their household. We comment 
where we feel this usage profile may impact the 
generality of our findings. 
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The research reported in this paper represents an attempt 
to establish the validity of the above position prior to 
the development of verification oriented research. We 
think this is necessary as our conceptual approach is 
somewhat different rom prior research in this area. 
Previous auempts to schematize or categorize software 
have oflen been done according to adult 
conccpiualizaiions (e.g., .lasscn, 1985; Taylor, 1980). 
According to our hypothesis, it is the child's perception 
and categorization of programs, particularly the 
interactive aspects, that determine how we should think 
about designing software (c.f. Shapiro, 1987; Turkic, 
1984; Winograd, 1986). Accordingly, our first research 
question is whether interactive formats have a strong 
enough influence on how children categorize software so 
that wc may speak of genres of inieraciivity. A second 
way our research differs from previous approaches is 
that some of the best works on the motivational aspects 
of computer software have tended to ignore differences 



Genres of Interactivity 

If children think of software in terms of interactive 
format, then it is reasonable to expect them to use 
similarities in interactive format across programs as a 
basis for categorization (cf. Chi, 1981). To decide if a 
cluster of programs coheres by interactive equivalences, 
without a cnsp definition of interactivity, could be a 
2 

dubious enterprise. However, we expected that this 



In fairness, Malone (Malone & Lepper, 1984) later added an 
interpersonal dimension of motivational variables thai capture 
the interactive appeal of a larger array of software. 
2„, 

We take mteractivity to mean something broader than, and 
prior to, a simple distinction between computer and "passive** 
media. As such, interactivity is a potentially unbounded 
concept ranging in examples from playing a game of tennis to 
reflecting on one's own thoughts.. In attempting to define 
interactivity, one could easily flounder in debates over false 
problems: Can true interactivity only lake place between two 
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problem of identification would not be too great. It was 
our xediction that students would create and label a few 
cat(;gories that unproblematically suggested 
interactivity. 

Fourteen children G)alanced for gender) each received a 
stack of 56 index cards that had an educational software 
fitle typed on one side. Children were directed to "sort 
the cards into piles of programs that are alike." No 
further instructions were given. Students were left to 
int^ret "alike" and decide how many piles to create. 
The groupings were recorded and children were asked to 
label each pile. 

A separate manipulation removed programs that a child 
had not actually used. We only consider the 26 
programs that were known by at least half of the 
students in both the sorting task and the motivational 
ranking task below. The sortings were subjected to a 
cluster analysis using the Rao-Russel coefficient of 
similarity and a complete linkage approach. The label or 
labels that occur most frequently over all the programs 
within a cluster are used as names for the resulting 
cluster. 

The analysis yielded seven clusters of programs that 
could be differentiated according to the labels assigned 
by the students. We give the label and our interpretation 
of the cluster along with the programs that were 
included The order of the programs end clusters listed 
below replicates the ordinal positions within the final 
clustering. For example, Bank Street Writer is most 
closely linked to LogoWriter and most distant from 
Path Tactics. 

Writing or Language Arts. These programs are 
typically seen as used for writing or typing practice. 
Bank Street Writer, LogoWriter, Paws, Writer Rabbit.^ 

Print-out Programs. This category is defined by the 
program's emphasis on the goal of printing something 
out Print Shop, Print Shop Companion, Time Liner. 

Puzzles. One student called this type of program 
"mind puzzles," This category revolves around deductive 
style reasoning. Gertrude's Puzzles, Puzzles & Posters. 

States or Social Studies. These programs all 
involve learning about the 50 states. However, the titles 
alone may define the category. Dataquest: The Fifty 
States, Game of the States, States and Traits, Coast to 
Coast. 



humans? Is drill and practice less or more interactive than 
inquiry*based software? Fortunately, our ultimate goal as 
researchers is not to come up with a necessary and sufficient 
definition of interactivity. Indeed, if there are concepts that are 
meaningful through the family resemblances of their 
examples, interactivity surely must be one. Rather, our goal 
is to organise discourse about those shared and differing 
attributes that characterize the many situations we call 
interactive. 



Interactivity and Motivation 

Adventure/Journey/Mystery. The anchoring 
attribute of this category appears to he computer 
simulation of the physical worid. One student made the 
astute distinction between mystery and adventure as one 
of solving vs. looking. These two types of programs 
are strong sub-clusters within this category. Odell Lake, 
Oregon Trails, Goodell Diamond, Treasure Hunter. 

Putting things together. Students have to 
construct •-nmething and test it Clock Works, Woodcar 
Rally. 

Math, This category includes many of the programs 
that deal explicitly with number. Within this cluster 
there is a tendency for drill and practice programs to be 
associated with one and another. MuttipUcation Puzzles, 
Subtraction Puzzles, Math Blaster, Circus Math, 
Quotient Quest, The Market Place, Path Tactics. 

There are two strong dimensions used to develop the 
categorizations: content area and interactive model. The 
three content area clusters, language arts, social studies, 
and mathematics, should not be very surprising. Given 
schools* strong partitioning of knowledge into curricular 
domains, one would expect children to bring their 
categorization of school experience to their 
categorization of software. This effect is probably 
heightened by Darien's use of the computer lab within 
class periods. 

Of greater interest for our purposes are the other four 
groupings, especially the printer, adventure, and 
building programs. From this small sample of children 
and programs, it can be seen that children are quite 
attentive to interactivity in their conceptualization of 
software. Each of these represents an identifiable way of 
interacting in and with the world: using tools, 
navigating, searching, and creating. One might easily 
imagine that less interactive but more frequently touted 
aspects of software, such as graphics quality or 
engagingness, could have been used to equate the 
software. Interactive models are not limited to action in 
the physical world, although this is the currently 
favored genre of fifth graders. There also appears to be a 
sub-cluster within the math grouping in which the 
student competes with an animated agent within the 
computer, thus modeling interpersonal interactivity. 
Undoubtedly, there are many other models of 
interactivity upon which computer programs do or will 
rely. 

Motivation ai I Types of Software 

If programs are equally motivating but rely on different 
models of interactivity, it is likely that the motivation 
originates from different sources. To investigate the 
relationship of motivation to interactive formats, we 
operationalized the motivational strength of a program 
to mean how often a child likes to use the program. 22 
fifth graders (balanced for gender) completed a 
questionnaire indicating how often (always, son.;.times, 
never, unknown) they would choose to use each of the 
56 programs in their free time. Below i§ the ranking. 
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from most m6tivating to least, of those programs that 
were known by at least half of the students. A score of 
three would represent the most motivating program. 

(2.50) Oregon Trail 
(2.50) Print Shop 

(2.45) Odell Lake 
(2.32) Bank Street Writer 
(2.21) Print Shop Companion 
(2.11) Wood Car Rally 
(2.10) Goodell Diamond 
(2.09)LQgoWritcr 

(1.9^ Hie Market Place 
(1.94) Path Tactics 
(1.94) Quotient Quest 
(1.92) £>ataquest: The Fifty States 
(1.92) Puzzles and Posters 
(1*83) Coast to Coast 
(1.82) Multiplication Puzzles 
(1.81) Time Liner 
(1.80) Gertrude's Puzzles 
(1.79) Game of the States 
(1.72) aock Works 
(1.69) States and Traits 
(1.68) Paws 
(1.65) Circus Math 
(1.58) Treasure Hunter 
(1.55) Math Blaster 

(1.46) Writer Rabbit 

Attitudes towards software programs appear to be fairly 
well defined across the population as there is a highly 
significant effect for program type (F = 6.9). There is a 
significant effect on motivation when clusters are treated 
as groups within an ANOVA (F = 7.75). Adventures 
and the programs that generated printer output were 
significantly preferred to the other clusters (Scheffe test 
atp < .05) but not to one and another. V/e will discuss 
the source of this preference in the next section. 
Interestingly, there is an interaction between gender and 
program type (F = 1.7, p < .03) but not between gender 
and cluster type (F = 1.45, p > .22). This suggests that 
it is not math or writing per se to which girls and boys 
object, but rather specific, and perhaps prototypical, 
implementations of the category of software. However, 
the sample is too small to be confident in a failure to 
find effects or to begin parceling out interaction effects. 

Oregon Trail and Print Shop are representative programs 
from the highly motivated clusters of adventure and 
print-out These two programs are extremely different in 
the sorts of interactions, goals, and sources of 
satisfactions a student can expect Although one might 
wish to claim that the same sorts of design elements 
make each motivating (e.g., an appropriate level of 
complexity or challenge), this seems to be the wrong 
level of analysis. We suggest that what makes each 
program so successful is the fit between the reasons the 
child wants to use the program and the interactive 
environment it creates. A program that includes 
navigational goals should create a spatial environment 
Being able to jump from menu to menu in Oregon Trail 



(as is the case in Print Shop) would destroy the 
interactive premise of travel. It would undermine our 
expectations of interactions within a spatial 
environment The program would lose much of the 
motivational power it gains by relying on our 
enjoyment and knowledge of navigaling in the yfad± 

The Source of Interactive Expectations 

If children perceive a |»ogram as an adventure game» 

they prefer to see a first-person suirogate carry out their 

3 

actions widiin the program's environment On the 
other hand, in a math, drill and practice setting, no such 
surrogate is expected (perhaps to move numbe» around 
on the screen), nor does it seem to provide motivational 
enhancement beyond the temporary effects of novelty. 
Why might this be the case? In simplest fashion, mudh 
of what is enjoyable in software borrows fiom what is 
recognized as an enjoyable activity in the real world 
Children have a prior nKxkl of intenictioa that includes 
surrogates (e.g., dolls) for fantasies. On the other hand, 
the activity and feedback involved with woricsheets - 
animated or not - have never included die notion of a 
surrogate. There would be a mismatch between their 
expectations of the worksheet genre of interactivity and 
its delivery witii fantasy elements. 

Our cenoal tenet is that children's motivation toward 
educational software can be understood to a lai^e extent 
tiirough the specifiable expectations children bring to 
the different genres of software. These genres of 
software are created and identified on die basis of the 
interactive models diat rest upon the child's experiences 
in the world. We maintain this position because 
children bring previous real-world interactive 
experiences to their understanding of the various formats 
of educational software. It is easy to see how they 
analogize or equate software to various real-world 
activities (e.g., "This irogram is like going on a trip.** 
This one is like taking a test." This one helps me 
make sometiiing.") They must import tiie goals of 
interactions in the world to find satisfaction within 
software. Software does not create goals and lasting 
motivation out of a vacuum - the computer is not that 
inherendy motivating. Ratiier, a piece of software will 
borrow the motivating goal structure and the attendant 
interactive environment diat supports a particular type 
of interaction in tiie world. The two most highly 
motivating clusters of programs, adventures and 
printing software are notable in their close linkage with 
tiie world One facilitates interaction witiiin a simulated 
physical world, the other assists social interaction 
through the creation of signs and cards. They each 
borrow their appeal through their different connections 
to real world interactions. Further, they each appear to 
be the most successful within their genre of software 
because they maintain fidelity to the previously 
experienced goals and environment of their interactive 
premise. 



This was revealed through the group discussions considered 
in the next section. 
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Interactivity and Motivation 



What Might an interactive Genre 
Look Like? 

If children are given an opportunity to create a program, 
they will create one that is consistent with, and 
motivating because of, the interactive genre that serves 
as its premise. To explore whether students would 
emphasize a coherent set of interactive elements and 
what those might be, the two authors conducted group 
interviews with two different sets of five children. The 
question that anchored the discussion was, '^If you were 
to make a computer program, what would it be like and 
what sorts of things would you include?" 

The responses of both groups were strikingly similar in 
their preference for adventure games. Each group 
distinguished these from mysteries and puzzles. These 
results are in accordance with the distinctions uncovered 
by the clustering task and the motivational rankings. By 
looking at the different suggestions the students made, 
we get a better sense of the coherence of the interactive 
model and collect evidence for an extensive definition of 
the adventure genre of interactivity. 

The children suggested that they wanted the 
manifestation of computer responses to be constrained 
by the same environment in which the student 
participates. We might call this a causally coherent 
model of interactivity in which consequences must 
reveal themselves through changes in the environment 
All actions, student and computer, must be similarly 
constrained by the environment. Decisions that are made 
within the program should change the environment in a 
causal fashion. The legacy of these changes remains in 
play throughout the course of the interaction. For 
example, forgetting to plant the com will become a 
damning mistake only when the food stores run out. 
Students pointed out several times that they did not 
want the program to give disengaged, omniscient, or 
external feedback; all consequences and computer 
reactions should be a natural result of the causal 
environment the program creates. 

A second interactive model preferred for the adventure 
game is a spatio-temporal model. For example, they 
liked an ability to move in any direction spatially. 
This can be contrasted with programs in which the child 
always moves forward. However, complete freedom of 
movement and choice is not a superordinate principle of 
interactive design. The children explicitly stated that 
they did not like the capability of going back in time to 
correct a mistake. Ideally, an adventure environment 
will replicate the advantages and constraints of a spatio* 
temporal worid 

Students repeatedly stated that they liked to explore and 
learn from their options and mistakes. They liked 
having to discover the sub-goals and emergent goals 
that must be satisfied to achieve the larger goal of the 
adventure. They wanted the program to be somewhat 
unpredictable but in a fashion consistent with the laws 
that govern the program's environment. They wanted 



optional levels of difficulty and novelty. In other words, 
they wanted to leara through interactions within a rich 
causal environment 

The adventure game relies on three basic interactive 
models that are derived from the child's interactions with 
the physical worid: causality, spatio-temporality, and 

exploration. The discovery or immersion learning 
style they have chosen finds a natural model within 
their own lives (Papert, 1980). However, it is important 
to note that this is only one genre of interactivity and a 
style of learning that grows from it It is important to 
imagine what sorts of social interactivity and learning 
styles will become supportable as computer power 
becomes more available in the classroom. 



Directions and Conclusion 

The methods of enhancing motivation within different 
interactive settings will differ depending on the goals 
the user generally attributes to certain types of 
interaction. Although the goal is ultimately to learn 
something, the use of interactivity can bring a much 
larger set of goals and expectations to bear, it is central 
to any interactive design that taese expectations are 
played upon. As a first step in delineating the nature of 
these expectations, it is worthwhile to think of the 
different types of interactions a user might expect 
within a given geiu-e of interactivity. A practical way 
for a designer to follow this prescription is to ask 
herself what sort of interaction does this computer 
instruction seem most like and what sort of role does 
the computer play in the interaction (e.g., partner, 
adversary, game board, worid, referee, reference tool, 
museum, etc.). Subsequently, the designer can consider 
what things make this sort of interaction most 
satisfying and work onward. 

The unique features of the new and highly plastic 
interactive te:hnologies have not been fully explored, 
either theoretically or empirically. Designers currently 
rely on models of development that are often out-dated 
or prematurely reductionistic. Much of the literature on 
motivation in instructional technology attempts to 
include all designs within large matrices and 
continuums of factors. They do not lake into account 
the qualitative differences of children's expectations in 
their encounters with insUnctional technologies. Much 
like film, there are genres of software (in use and 
developing) that capitalize on the child's motivations in 
fundamentally different ways. Our future research will 
attempt to prove the interrelation of interactive format 



There were other elements of adventure games that the 
children mentioned such as codes, fantasy, and being able to 
choose the characteristics of their surrogates. There were also 
things that children claimed were unimportant: Gn^hics, high 
action, points, winning, and, sound. It is worth noting that 
some of the things excluded are precisely those features that 
are mandatory for the success of other genres of interactivity. 
It would be informative to ask students to design the ideal drill 
and practice program as a source of contrast. 
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and motivaUon through an experimental design. 
However, the next step is to continue the discourse with 
children about the different interactive elements and their 
groupings into genres. It is here that we will fmd the 
best ingredients for coherent and motivating design 
recipes. 
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COLLECTION OF DATA 
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Abstract 

An integrated hypermedia training curriculum, utilizing 
computers and interactive videodiscs, is being developed 
at Lehigh University for staff and personnel of the 
Cities in Schools, Inc., a national program addressing 
the critical problem of coordinating the delivery of 
human services to at-risk youth and their families. 
Cities in Schools programs address the problems of 
dropout prevention, drug abuse prevention, teen 
pregnancy, and teen violence. Extensive information is 
needed on how trainees utilize the integrated hypermedia 
materials, how they feel about using them, and what 
they learned. The formative evaluation component of 
the project seeks to provide this information efficiently 
and unobtrusively by using techniques and procedures to 
provide transparent automatic collection of data relating 
to use of the curriculum materials. This paper will 
describe techniques and procedures in use and under 
development to provide curriculum designers and 
developers comprehensive information on how trainees 
use the training curriculum, how they feel about the 
training, and what they learned from it. The extension 
of these techniques for utilization in a formal research 
program o*^ hypermedia will also be discussed. 

The Cities In Schools Program 

Cities in Schools is a national non-profit corporation 
with headquarters in Washington, DC, and regional 
office^ in Pittsburgh, Atlanta, Chicago, Los Angeles, 
and Austin, Texas. Recognized as a national leader in 
promoting and facilitating the coordinated delivery of 
existing health, social, education, and other support 
services for at-risk youth and their families. Cities in 
Schools seeks to develop private/public partnerships 
designed to connect appropriate human services with at- 
risk youth in addressing such critical issues as school 
attendance, literacy, job preparedness, teen pregnar.cy, 
drug and alcohol abuse, teen suicide, and school 
violence. Cities in Schools is in the process of 
establishing and providing training and technical 
assistance to a national network of centers to coordinate 
the delivery of multiple, existing social services to at- 
risk students and their families. 

Current and Future C.I.S. Hypermedia 
Project Activities 

The Cities in Schools Curriculum Project began in 
February, 1989, and completed Phase One of the project 
(the development, testing, and related staff training for 
training modules on history, organization, and the 
Cities in Schools replication process) in August, 1989. 
The experiences gained in developing the Phase One 



Cities in Schools hypermedia materials and preliminary 
results of their use in actual training have demonstrated 
the value of the approach taken. Two additional factors 
have influenced the future direction of the project First, 
it was felt both by Cities in Schools staff and the 
hypermedia development team that two of the primary 
objectives of Cities in Schools training were to convey 
the philosophy of Cities in Schools and to develop an 
emotional as well as intellex:tual commitment to its 
goals. This philosophy embodies a commitment to 
decentralization and local community control. Cities in 
Schools is described by William Milliken, its president 
and one of its founders, as "a mission in the process of 
becoming a program" (Milliken, 1989). It was felt that 
motion video was necessary in order to convey the 
philosophy effectively and to promote the desired 
commitment. The second influencing factor is that 
Cities in Schools has, since its inception, made 
extensive use of videotape in documenting its programs 
and in disseminating information about them Thus, a 
considerable amount of videotape materials about the 
program already existed. 

For these and other reasons, it was decided that the next 
phase of the Cities in Schools hypermedia project 
would involve the redesign of the entire training 
curriculum as an integrated hypermedia-based 
curriculum. This curriculum will utilize the full range 
of multimedia materials, including motion video, as 
well as print materials. 

This phase of the project calls for a year-long project 
beginning in August, 1989. A complete review of the 
Phase One ':urriculum materials will be conducted, and 
an overall plan for the goals, scope, instructional 
approach, and general philosophy of the integrated 
hypermedia curriculum will be developed, along with a 
list of available print, photographic, and video resources 
which mif^ht be included. During the spring of 1990, 
.idditioncii video materials will be produced. These will 
l^. combined with existing vidci. materials and 
transferred to interactive videodisc. At the same time the 
hypermedia software will be rewritten to integrate the 
videodisc materials into the overall curriculum. The 
complete curriculum will be completed and tested during 
the summer of 1990 at the proposed Cities in Schools 
National Center for Partnership Development joindy 
sponiored by Cities in Schools, Inc., the College of 
Education at Lehigh University, and Lehigh's lacocca 
Institute. 



393 



Story and Harvey 



The Need for Formative Research 

As Jonassen has stated, hypermedia is a rapidly 
expanding technology that offers considerable 
instructional potential (Jonassen, 1988). However, 
significant problems with hypermedia challenge both 
authors and users. Kearsley points out a number of 
formatting and organizations issues (Kearsley, 1988). 
There is a need for complete information on how the 
user uses the hypermedia materials, how the user 
responds to the hypermedia materials, and what is 
learned from the hypermedia materials. This information 
is important to the designers and developers for use in 
improving existing hypermedia materials and in 
designing future materials. These formative research 
activities can also provide the foundation for future 
more systematic and more formal research on using and 
learning from hypermedia. 

An efficient method to collect needed information will 
utilize the hypermedia technology itself, incorporating 
data collection into the actual hypermedia training 
modules. Data collection methods used should be 
transparent to the user to minimize distracting the user 
or interfering with learning from the hypermedia 
materials. To meet the needs of the Cities in Schools 
Hypermedia Development Project for formative 
evaluation data, a series of such procedures have been 
developed or are under development by project staff. The 
following section will present the types of informatio^i 
to be collectca, explain why that information is needed, 
and briefly describe tnc techniques used to collect that 
information. These techniques were developed through 
custom programming in HyperCard for Macintosh 
computers. Advanced textbooks on HyperTalk 
programming [for example, Goodman, D. (1988), 
Goodman, P. (1989), Michel, (1989), Shafer (1988), 
the Waite Group (1989)) were used as guides for 
developing these techniques. 

Transparent Automatic Data Collection 
Techniques 

Type and Sections of Training Curriculum 
Used. Users will be exposed to a variety of curriculum 
modules or to different versions of the same module. 
The hypermedia training materials automatically record 
the particular modules or version used for later cross- 
reference to user achievement, user attitudes, and other 
variables. 

User ID or code. It is essential to keep track of users 
in the evaluation procedures. To facilitate this, a unique 
identification number is automatically assigned to each 
user the first time he or she uses the hypermedia. If 
needed, at this point the user is also asked to enter 
additional demographic data such as age, sex, highest 
grade completed, etc. This information is automatically 
stored with other data for each user, and can then be used 
to identify aggregate information about test groups. Th^^ 
procedure guarantees anonymity fOi the user while 
providing efficient tracking of individual activity. 



For some applications, users have been given 
preassigned identification numbers. In these cases, the 
hypermedia system will prompt for this information, 
and will not proceed the user enters the correct 
information. Pressing the [Return] key will have no 
effect until the task is completed. 

Data Related to Time on Task. For each session, 
data on the time begun and time ended and the date are 
needed. This information will be useful for recording 
time on task and for tracking testing or training 
schedules. The date and time are recorded immediately 
after recording the user ID. The time the user quits the 
session is recorded before termination of the session 
when user presses a [Quit] function. 

Score on Achievement of Learning 
Objectives. It is necessary to keep the raw score of 
the users as they complete the built-in tests for 
achievement of training objectives. This information is 
needed to perform various analyses to determine how 
much learning has taken place. When the user presses 
the [SCORE] function the score of the test is recorded 
to the screen and to a hidden text field. When the user 
leaves the [TEST] screen the score is recorded to a 
hidden text field and saved for later analysis. 

Item Analysis of Built-in Achievement 

Tests. Validation of built-in achievement tests requires 
recording which questions were responded to correctly or 
incorrectly. When the user presses the [SCORE] 
function or leaves the [TEST] screen the procedure 
records which questions were aaswered incorrectly or not 
answered. The procedure will also record which response 
was chosen for each question for later use in item 
analysis. 

Sequence of Using Material. From the 
perspective of learning theory, hypermedia should 
improve learning because it focuses attention on 
relationships between ideas rather than on isolated facts 
(Kearsley, G., 1988). To investigate this perspective it 
is necessary to identify and to record the sequence in 
which a user progresses through hypermedia materials. 
When the user accesses a screen, a hidden text field on 
that screen is marked to indicate the screen has been 
accessed. The ID number, card number, or card title is 
also recorded in a text field on a separate card, generating 
a list of materials used in ihc order in which they were 
accessed. This process continues until the user presses 
the [QUIT] function. 

Amount of Hypermedia Material Used. 
Hypermedia materials contain a vast amount of 
information. This information may be in the form of 
text, graphics, still photographs, or video.. It is 
necessary to establish how much of the total amount of 
information available was used by the user. This 
information can be used to determine whether there was 
not enough material available to the user or conversely 
whether too much material was available, thus 
overwhelming the user. 
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The amount of hypemedia material used is recorded in a 
hidden text field on a separate tracking card. The ID 
number or title of a screen is recorded each time that 
screen is accessed, and the screen itself is marked as 
having been accessed. When user presses the [Quit] 
function, a list of screens which were not accessed* the 
total number of screens not accessed, and the percentage 
of the total amount of hypermedia accessed is recorded 
automatically. 

Amount and Effectiveness of Use of 
Navigational Aids. As Jonassen has stated, one of 
the most obvious user problems is navigation through 
hjpcrmedia (Jonassen. D.. 1988). How does a user use 
navigational aids to navigate through hypermedia 
material? This technique will enable a developer to "see" 
how the user utilizes navigational aids. Every time a 
specific navigational aid is used this fact is recorded. 
The tracking procedure can be set to record each time the 
navigational aid is used or the total number of times it 
is used. The procedure also records from which screen 
the navigational aid was activated and/or to which screen 
the user linked. When user presses the [Quit] function 
the hidden text field is then checked and the information 
reoortkxL 

Degree of Exposure to Information Related 
to Specific Objectives. It is important that a user 
be exposed to all of the relevant hypermedia material for 
a given objective. It is also important for developers to 
know how much of a given amount of information the 
user has been exposed to. This may be used to interpret 
a user's score or to determine the minimum amount of 
exposure to material for a given objective needed to 
meet training objectives. To record this information 
each screen of information is tagged with a hidden text 
field that identifies the related objective. When the 
screen is accessed a text field on a separate record card 
records the screen ID and related objective. When user 
presses the [Quit] function the tracking information is 
compiled and recorded. The screens that have not been 
tagged as having been accessed are also recorded along 
with their related objective number or ID. 

Provide for Open-Ended User Feedback. While 
developers have tried to foresee and address many of the 
problems that users may encounter, there is still the 
need for constructive user feedback. The user has the 
option to enter a NOTE/QUESTION screen at any point 
in the hypermedia. This enables the user to record any 
questions or concerns he or she might have concerning 
this particular training module. These questions will 
then be recorded at the end of the session. 

Generating Summary Reports. It is of no benefit 
to acquire large amounts of information and not use it. 
The purpose of the techniques described in this paper is 
to gather useful information using mechanisms 
transparent to the user and then present that information 
to trainers and curriculum developers in a useable form. 
The information gathered by these procedures and 



techniques can be recorded as a separate document foi 
each factor nteasured or recorded as one large document 

Relationship to Formal Research. The 

procedures and techniques described in this paper have 
been developed to guide and to support the development 
of effective hypermedia training materials. Emphasis 
has been placed on gathering useful information quickly 
and unobtrusively. The techniques developed, however, 
are also potentially useful for more formal research 
invr igations. The experience in developing and 
impi.<^menting the formative evaluation component of 
the Cities in Schools Hypermedia Development Project 
will provide a fum foundation for later formal research 
investigations. Project staff are now developing a 
research agenda for systematic investigation of how 
specific hypermedia factors relate to learning and user 
attiuide. 



Conclusion 

Hypermedia technology is a new and exciting medium. 
As with all technology, effective use of hypermedia 
depends on careful and systematic investigation of how 
the technology is used and of the effect of changes in 
specific attributes of the technology on leaming, user 
attitudes, and ease of use. Formative evaluation 
activities attempt to cany out these systematic 
investigations in the context of an ongoing technology 
development project, while more formal research 
projects attempt to answer similar questions in a more 
systematic and longer term way. Hypermedia is unique 
in the amount of data collection techniques that can be 
incorporated into the hypermedia materials being 
evaluated. It is hoped that the techniques described in 
this paper will promote the more efficient and more 
comprehensive collection of data on hypermedia. These 
and continuations of these activities, we believe, will 
contribute to grecitcr understanding and more effective 
use of hypermedia technology. 
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Abstract 

The development of an approach for the training of 
Georgia Department of Corrections managers has 
evolved from the findings of an objective empirical 
study. The commission of this study, Management 
Development Training Evaluation for the Georgia 
Department of Corrections, produced some new 
correctional training issues not previously studied. 
Based on the results of one empirically-based 
component of the study, the project team discovered 
differences in the learning styles of the sample large 
enough to dictate the use of different instructioral 
approaches with particular groups of trainees. Results of 
this study also contributed to the development of a 
learning environment model that will allow correctional 
trainers to design and implement instructional activities 
for particular groups of trainees. Furthermore, the 
findings of this study will enable the Department to 
develop new strategies to proceed toward a training 
program that will best meet the needs of the training 
population. 

Currently, the Georgia Department of Corrections is 
experiencing unprecedented growth in the numbers of 
staff, facilities, and offenders under supervision. Of 
critical interest to the Department is the need to train 
and develop staff at all levels of management in the 
most effective and efficient manner. Consequently, on'* 
question this study attempted to address was whether the 
instructional strategy in place at the time this study was 
commissioned was the most effective and efficient in 
accomplishing the Department's goal of having a trained 
and qualified pool of managers. Therefore, the primary 
purpose of this study was to examine the 
appropriateness and applicability of using a multimedia 
instructional strategy in implementing the Management 
Development Training Program within the Georgia 
Department of Corrections. 

One component of this study was designed to reveal 
whether or not there were any statistically significant 
differences in the learning styles of the target training 
population (that is, between the supervisors, middle 
managers, and upper managers within the Facilities 
Division and Probation Division). Another objective 



was to identify which training media and instructional 
methodologies were most suited to the dominant 
learning styles of the training population. 
Consequently, results of this study will serve as a guide 
in evaluating, recommending and designing learning 
experiences and techniques of multimedia delivery that 
best match students' styles with the learning objectives 
of individual training programs. 

Method 

Subjects. The target population in this study 
consisted of individuals employed by the Georgia 
Department of Corrections in supervisory and 
managerial positions. Subjects in this population 
represent the full range of managerial job classifications 
as identified by the Staff Training and Development 
Section. 

The population of managers consisted of 890 
individuals (69 job classiflcations) grouped as 
supervisory management, middle management, upper- 
middle management, upper management and executive 
management. Although the top managers within 
Cenural Office executive management were not one of 
the focal management types to be examined, surveys 
were reviewed and approved by them for distribution to 
their subordinates. In fact, several were completed by 
these managers and returned to the project team.A 
stratified proportionate random sample of the population 
was used. The population was divided into five strata 
using criteria developed by the Staff Training and 
Development Section of the Georgia Department of 
Corrections. The underlying rationale was to divide the 
population into groups based on the guidelines used to 
determine the composition of the training classes 
themselves. 

In order to determine the size of the sample required to 
estimate the population proportion, a pilot study was 
conducted. Application of the general formula for 
estimating population totals (Jaeger, 1984) indicated 
that an overall sample size of 302 was required in order 
to estimate population means within a .75 errcr limit 
and 95% confidence level. 

In order to ensure an adequate response rate (return rate 
for the pilot study was 75%), 402 supervisory and 
managerial personnel were randomly selected with a 
roughly proportionate number of subjects from each 
stratum (the two smallest strata were oversampled). The 
sample included 133 supervisors, 128 middle managers, 
35 upper-middle managers, 91 upper managers, and 15 
executive managers. This sample represents 45% of the 
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population of supervisory and managerial personnel in 
the Georgia Department of Corrections. 

Instrumentation. One questionnaire in the survey 
packet was Kolb's LSI, a widely used learning style 
inventory that allows learners to be classified by their 
dominant or preferred learning style and corresponding 
learning environment derived from experiential learning 
theory. Responses to the LSI were used to determine 
whether there were any differences in the learning styles 
of the training population. The LSI is designed to be 
self- administering with simple instructions and an 
example. The four basic scales all have very good 
internal reli«i)ility as measured by Cronbach's alpha 
(.82, .73, .83, .78) (Kolb, 1984). Despite the 
weaknesses suggested by some research concerning 
certain properties of the LSI, work by other researchers, 
such as Karrer (1988), state that learning style 
instruments other than the Kolb are even more suspect. 

Experiential learning is conceived as a four-stage 
learning cycle. At any stage of a learning cycle, different 
abilities are required of the learner concrete experience 
(CE) abilities, reflective observation (RO) abilities, 
abstract conceptualization (AC) abilities and active 
experimentation (AE) abilities. Learners input 
information either from experience (CE) or from 
abstractions (AC) and process information either from 
internally reflecting on the experience (RO) or 
externally acting upon the conclusions which have been 
drawn (^). 

Presentation of the Data and Results The distribution of 
the survey packet and the reminder postcard sent two 
weeks later brought an 87% return of completed 
questionnaires. A total of 333 acceptable surveys (83% 
returned and found usable) were used in the data 
analysis. 

Data Analysis of LSL The computer scoring 
of the Kolb Inventory was in accordance with 
the scoring procedure from the LSI Manual (Kolb, 
19»6). Reliability estimates using Cronbach's 
coefficient alpha yielded the following strong reliability 
coefficients: .75 for Concrete Experience, .79 for 
Reflective Observation, .81 for Abstract 
Conceptualization, and .84 for Active Experimentation. 
Since these reliability estimates were as high if not 
higher than those reported previously in the research 
literature, \hf project team determined that the LSI data 
collected in this study was highly suitable for the 
research application described herein. 



Management Training and Learning Styles 

In accordance with the scoring procedure from the 
Learning Style Inventory Technical Manual, scores on 
the four scales of the LSI (CE, RO, AC, AE) were 
combined to equal the two learning dimensions which 
identified the dominant learning style for the training 
population. The two major learning dimensions were 
Abstract-Concrete (AC-CE) and Active-Reflective (AE- 
RO). 

One important area examined in this study focused on 
whether or not there were any learning style differences 
between field personnel who work in the Facilities 
Division (prisons) and those who woric in the Probation 
Division (programs). The Department v/anted to 
determine whether integrating staff from these 
operational divisions into the same training programs 
would enhance or detract from the learning experience in 
the classroom. The "Facilities Division" includes 
supervisors and managers who are employed in the 
typical "prison institution" and the "Probation 
Division" includes supervisors and managers who work 
with a probation type of "program." Field supervisory 
and managerial personnel who work in these divisions 
make up the majority of the training population (as 
well as the sample). Table 1 presents the means and 
standard deviations on the four scales of the LSI by 
division type of field personnel. 

Table 1 indicates rhai some differences may exist in the 
abilities of Facilities and Probation Division personnel 
to perceive information. Facilities Division personnel 
scored higher than Probation Division personnel on 
taking in information concretely (CE mean = 26.06 and 
24.65, respectively). Furthermore, Probation Division 
personnel scored higher than Facilities Division 
personnel iii perceiving information abstractly (AC 
mean = 32.03 and 30.26, respectively). Table 1 also 
indicates smaller differences in how Facilities Division 
and Probation Division field personnel processed 
information (AE mean = 31.86 and 30.62, RO mean = 
32.70 and 31.84, respectively). 

Multivariate analyses of variance (MANOVA) were used 
to assess differences in mean scores along the two 
learning style dimensions. These different analyses 
examined differences between division types (Facilities 
and Probation). In addition, differences in management 
types within both divisions were also investigated. 



Division Type of 
Filed Personnel 


Leamine Stvlc Scale Scores 






CE 


RO 


AC 


AE 




n 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


Facilities 


194 


26.06 


6.40 


31.84 


6.91 


30.26 


5.92 


31.86 


7.16 


Probation 


60 


24.65 


6.62 


32.70 


6.61 


32.03 


6.88 


30.62 


7.26 
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Due to the exploratory and non-experimental nature of this study, a .10 level of signincance was used throughout 
these analyses of variance. The univariate F values indicate results which would have been produced if each dq)endcnt 
variable had been analyzed in isolation. 

The goal in the first MANOVA was to ocicrmine whether scores on the two separate learning dimensions were 
different for the Facilities Division field personnel and Probation Division field personnel. Results are presented in 
Table 2. 

Table 2. Multivariate Analysis of Variance (MANOVA) Comparing Learning Style Dimensions by Division Type 
of Field Personnel 



Dependent Variable 


MS 


Enor 
MS 


F Value 


ACCE 


464.09 


108.28 


4.29** 


AERO 


202.86 


145.89 


1.39 


Multivariate Test of 
SicnificancefPillais) 






1.37* 



•p < .10.; ♦♦p < .05. 

Note. Multivariate normality and homogeneity of variance-covariance; matrices assumptions were tested and 
robustness ensured. Adjustments for unequal sample sizes among the different management types were also 
considered in the analysis 

Results of the MANOVA procedure indicated that learning style preference as measured by the LSI differed 
significantly among the sample of Facilities Divijsion and Probation Division supervisors and managers (F « 131, p 
= .08). The univariate F tests revealed significant differences in the AC-CE learning dimension (F « 4.29, p « .04) 
but not the AE-RO learning dimension (F = L39,p = .24). 

Of further training-related interest was the examination of differences in the management types within both the 
Facilities Division and Probation Division managerial personnel. Due to the difference between the management 
structures of the divisions (the Facilities Division has a more extensive middle nuuiagement structure), two separate 
analyses were necessary. The first analysis pertabed to field supervisors and managers within the Facilities Division. 
Thr following table presents the means and the standard deviations on the four LSI scales of Facilities Division field 
personnel by management type. 



Table 3. Means and Standard Deviations of Learning Style Abilities of Facilities Division Held Personnel by 
Management Type 



Management 
Type 


LeamioK Style Seal 


le Scores 






CE 


RO 


AC 


AE 




n 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


Upper 


22 


27.86 


6.74 


28.73 


5.40 


27.77 


6.17 


35.73 


5.39 


Upper Mid 


26 


25.31 


6.72 


32.65 


6.16 


32.81 


6.18 


29.23 


6.50 


Middle 


77 


25.91 


6.92 


32.03 


7.85 


30.52 


5.96 


31.56 


8.29 


Supervisor 


69 


25.94 , 5.55 


32.30 


6.33 


29.81 


5.42 


31.94 


6.03 



Table 3 indicates that some differences exist in the abilities of Facilities Division supervisors and managers to 
receive, as well as process, information. The largest differences appear to be between upper managers and the other 
managcrnent types. For example, upper manai^crs score higher than the other management types on perceiving 
information concretely (CE mean = 27.86), and lower than the other managerent types when perceiving information 
abstractly (AC mean = 27.77). Upper management Facility Division field personnel also score higher than the other 
management types in processing information actively (AE mean = 35.73), and lower than the other management 
types in processing information reflectively (RO mean = 28.73). A MANOVA was then used to discover whether 
scores on the two separate learning dimensions were significantly different for the Facilities Division management 
types. The results of the MANOVA procedure are presented in Table 4. 
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Table 4. Multivariate Analysis of Variance (MANOVA) Comparing Learning A Style Dimensions of Facilities 
Division Personnel by Management Type 



Etepcndcni Variable 


MS 


Enor 
MS 


F Value 


ACCE 


236.23 


98.40 


2.40' 


AERO 


469.49 


141.43 


3.32** 


Multivariate Test of 
SiKniflcance (Pillais) 






2.40** 



*p < .10.; ♦♦p < .05. 

Note. Multivariate normality and homogeneity of variance-covariance matrices assumptions were tested. 
Robustness was ensured with the AC-CE dimension but not the AE-RO dimension. Results pertaining to the AERO 
dimension should be interpreted with caution. Adjustments for unequal sample sizes among the different management 
types were also considered in the analysis. 

Results of the MANOVA procedure indicate that learning style preference, as measured by the LSI, differed 
significantly among the Facilities Division management types (F = 2.40, p = .027). In this case, the univariate F 
tests revealed significant differences in the AC-CE learning dimension (F = 2.40, p ^ .069) and the AE-RO learning 
dimension (F « 3.32, p = .021). 

Since the learning style dinnensions of the four management types were found to be significantly different, a multiple 
comparison test was used to determine what significant pairwise differences existed. Due to the exploratory nature of 
this study, a liberal multiple comparison test (Fisher's LSD) was used to determine which management types were 
significantly different at the .05 level of significance. Table 5 summarizes the result* of the multiple comparison test 
by indicating where a statistically significant difference occurred. 



Table S. Summary of Results of Multiple Comparison Tests on Learning Style Dimensions 



LeaminK Dimension 


Facilities Division ManaKement Type 




Upper 


Supervisor 


Middle 


Upper Mid 


AC-CE 


ns22 


n»69 


n»77 


n = 26 


Mean 


-.09 


3.87 


4.61 


7.5 




Middle 


Supervisor 


Upper Mid 


Upper 


AE-RO 


n»77 


n»69 


n«26 


n = 22 


Mean 


-.49 


-.36 


-3.42 


7.0 



Note. Any management types underlined by a common line did not differ significantly. Management types are rank- 
ordered with respect to mean scores arranged in order of magnitude from the smallest to the largest. 



Table 5 indicates that, with respect to the AC-CE learning dimension, there were significant differences in the 
learning styles of upper management and upper middle management, as well as between upper management and 
niiddle numagement Facilities Division personnel. With respect to the AE-RO learning dimension, significant 
differences were noted between upper and upp^ middle managemciit, upper and middle management, and upper and 
supervisory management Facilities Division personnel. 

The fcllowing analyses pertain to Probation Division field personnel. Table 6 presents the means and the standard 
deviations of the four LSI scales for Probation Division field personnel separated by management type. 

Table 6 indicates that small differences existed in the learning abilities of Probation Division upper management and 
supervisory management field personnel. The most apparent difference appeared to be between upper managers and 
supervisors in perceiving information concretely. Probation Division upper managers scored higher than supervisors 
in perceiving information concretely (CE mean » 25.9.^ and 22.41, respectively), and lower than supervisors in 
perceiving information absuactly (AC mean 31.37 and 33.18, respectively). Upper management Probation 
Division personnel also scored higher than Probation Division supervisors in processing information actively (AE 
mean s 30.71 and 30.45, respectively) and lower than supervisors in processing information reflectively (RO mean = 
31.97 and 33.96, respectively). It is interesting to note that these same differences, though not as large, were 
discernible with respect to the learning abilities of upper management Facilities Division personnel and their 
subordinates. 
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Table 6. Mcafts and Standard Deviations of Learning Style Abilities of Probation Division Field Personnel by 
Management Type \ 



ivwnagcnicni 
Type 


Lcaminc Style Seal 


e Scores 






CE 


RO 


AC 


AE 




n 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


Upper 


38 


25.95 


6.41 


31.97 


6.64 


31.37 


6.80 


30.71 


7.30 


Supervisor 


22 


22.41 


6.52 


33.96 


6.52 


33.18 


7.04 


30.45 


7.35 



A MANOVA was used to determine whether scores on the two learning dimensions were significantly different for 
the upper and supervisor Probation Division management types. The results of this MANOVA procedure are 
presented in Table 7. 



Tiibic 7. Multivariate Analysis of Variance (MANOVA) Comparing Learning Style Dimensions of Probauon 



Dependent Variable 


MS 


Error 
MS 


F Value 


ACCE 


399.06 


129.02 


3.09* 


AERO 


69.71 


145.08 


.48 


Multivariate Test of 
Sicnificance (Pillais) 






1.77 



•p < .10 

Note . Multivariate normality and homogeneity of variance-covariance matrices assumptions were tested and 
robustness was ensured with both the AC-CE and the AE-RO learning dimensions. Adjustments for unequal sample 
sizes among the two different management types were also considered in the analysis. 

Results of the MANOVA procedure indicated that learning style preference as measured by the LSI did not differ 
significantly among the sample of Probation Division upper management and supervisory management personnel (f 
= 1.77, p = . 179). However, one univariate F test indicated a significant difference in the AC-CE learning dimension 
scores (F = 3.09, p - .084), while the other indicated no significant difference with respect to the AE-RO learning 
dimension (F = .48, p = .49). 



Recommendations 

The data obtained from the LSI can be used in many ways in 

training related matters. Perhaps the most obvious is for the purpose of designing instruction prior to the 
implementation of training for each particular group of trainees. Using the results summarized in this report, 
methodologies and media approaches can be planned to nwct the needs of the majority of training participants. 

HUMANITIES BASED - SOFT 
(non-paradigmatic) 



AFFECTIVE 
DOMAIN 



PRACTICAL 
BASED 
(applied) 



BEHAVIORAL 
DOMAIN 



PERCEPTUAL 
DOMAIN 



THEORY 

BASED 

(pure) 



SYMBOLIC 
DOMAIN 



MATH BASED - HARD 
(non-paradigmatic) 

Figure 1. Domains in which Learning Environments vary. 
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Matches between learning styles and measures of 
environmental complexity could lead to higher learning 
outcomes and more learner satisfaction. (Fry, 1978). 
Research by Biglan (1973) and Kolb (1975) has 
produced a model of four distinct domains or areas in 
which learning environments can vary. These domains, 
presented in Figure 1, have been labeled Affective, 
Perceptual, Symbolic, and Behavioral. 

An affective domain includes situations where that 
which is being learned is non -paradigmatic, soft and 
humanities-based. Figure 1 indicates that this can be 
contrasted with the symbolic domain, where learning is 
hard, science-based and paradigmatic. On the other hand, 
*4he perceptual domain, which is identified as being pure 
and theory based> emphasizes the exploration and 
understanding of basic relationships between causes of 
events and phenomena. The perceptual domain is 
contrasted with the behavioral domain, where the 
emphasis is on practical use or application rather than 
understanding or theory. (Fry, 1978) Specific descriptive 
indicators for each of these learning environments, as 
identified by Fry (1978) and by our review of the 
literature, are presented in Table 8. 

FromSthe descriptors of learning environments in Table 
8, we can sec that the environments which best support 
the various learning abilities are summarized as follows: 

• symbolic environment supports learners whose 
best learning ability is abstract 
conceptualization; 

• perceptual environment supports learners 
whose best learning ability is reflective 
observation; 

• affective environment suRwrts learners whose 
best learning ability is concrete experience; 

• behavioral environment supports learners 
whose best learning ability is active 
experimentation. 

By examining earlier findings about the optimal 
learning ability (determined by er^amining the 
differences in the learning dimensions) of training 
participants, approximations can be made in 
determining the most appropriate learning environments 
for a particular group of management types. Table 9 
indicates the most appropriate and second most 
ai^opriate learning environment for each management 
type of both the Facilities Division and Probation 
Division personnel. 

Table 9 indicates that the most appropriate learning 
environment for upper management field personnel in 
the Facilities Division is a beha.iorJ environment (sec 
Table 8 for a description of a behavioral learning 
environment) that allows upper managers to learn with 
their best learning ability, active experimentation. In 
addition. Table 9 indicates that the symbolic 
environment (see Table 8 for a description of a 
symbolic learning environment) most closely 
approximates the needs of the rest of the Facilities 
Division and Probation Division field personnel, since 



they have varying de^^ of abstract conceptualization 
as Uieir strongest ability By custom designing courses 
using the most appropriate learning environment for a 
particular management type, the highest levels of 
efficiency can be obtained. 

Based on the findings summarized herein, training staff 
in the Georgia Department of Corrections may wish to 
modify the design, method of presentation and 
instructional activities of certain courses in the 
Management Development Training Program. For 
example, when implementing a course for upper 
managers in the Facilities Division, the learning 
environment should be primarily behavioral in nature^ 
allowing these managers to utilize their best learning 
ability and preferred instructional activities (active 
participation types of activities such as case studies, 
simulations). However, when implementing this same 
course, (or example, with supervisors in the Probation 
Division who have abstract conceptualization as their 
strongest learning ability, the course should be revised 
to suit the Probation supervisors preference for abstract 
information and instructional activities that support 
their need to practice their use of theory and analytical 
skills. 

The present researchers believe that trainers and 
instructional designers must become aware of the 
predominant learning abilities of their training 
populition. Moreover, they should become familiar 
with the most appropriate learning environments that 
best support a particular learning ability. As a result, 
trainir^ prpgrams can then be designed and implemented 
to optimize their success. With some basic 
understanding of the learning styles of a training 
population, learning environments can be planned, 
irstructional methodologies chosen, and media selected 
u» best meet the needs of the learner. Recently, staff 
trainers, using recommendations from this study, 
modified the instructional activities and presentation 
methods of a management training course for Facility 
Division upper middle level managers. Though a formal 
evaluation has yet to be completed, comments by the 
students indicated that the program was highly 
successful: ''the best training Tve ever had...I learned a 
lor (Allen, 1990). 

Summary 

The LSI data obtained from this study's training 
participants can be used in numerous ways by training 
staff. Methodologies and media approaches can be 
planned tu meet the needs of the training participants. 
Furthermore, •he data can be used during the training 
session to makf revisions in the presentation of 
material or assignment^; to participants. 

Based on information contained in this report pertaining 
to learning styles, learning environments, and 
instructional activities, trainers can utilize different 
approaches for different groups of training participants 
when delivering the instructional material contained in 
the new Management Development Training Program. 
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Table 8> Descriptive Indicators of Learning Environments 





TYPE OF ENVIRONMENT 


CATEGORY OF INDICATOR 


AFFECTIVE 


PERCEPTUAL 


Objectives: 


Basic purpose is to experience an 
event or activity, to be aware of one's 
feelings while going through it. 


Basic purpose is to understand 
something, to know how and why 
things relate to each other. 


Principal focus or source of 
information being dealt with: 


Information is here and now, in the 
form of personal feelings, values, 
opinions, ideas, etc. 


Information is derived from 
examining how and why something 
occurs, focusing on the process, 
reviewing past events. 


Nature of feedback and rewards: 


Feedback is personalized, based on 
each individuates own needs and 
learning goals. 


Learner determines criteria for 
evaluation; and is left to determine 
own criteria for performance 


Nature of learner's role: 


Learners freely express personal 
feelings, opinions and values 
concerning topic or activity in which 
they are engaged. 


Learners are encouraged to observe, 
listen, write, think, discuss, etc. in 
order to deter mine meaning and 
relevance of subject matter for 
themselves. 


Teacher Role: 


Teacher models the desired or 
appropriate behavior such that learners 
learn by example and through relating 
(identifying) with the teacher. 


Teacher is non*direcuve, reflective, 
and nonevaluative and teaches by 
helping the learner to discover his her 
own perspectives, insights. 


Instructional Activities and 
Enrichment Components: 


Laboratories, observations, primary 
text reading, simulations, games. Held 
woric, trigger filni5 readings, problem 
sets, examples, structured activities. 


Journals, logs, discussion, 
iHainstorming, thought questions, 
rhetorical questions, role playing, 
causal models, advance organizers, 
compare contrast 


Learning Ability Best Supported: 


Cbncrcte Experiencer 


Reflective (X)server 






TYPE OF ENVIRONMENT 


CATEGORY OF INDICATOR 


SYMBOLIC 


BEHAVIORAL 


Objectives: 


Basic purpose is to solve a problem , 
to obtain a solution through use of 
theory and analytical skills. 


Basic purpose is to apply knowledge 
to practical problems such as one 
would experience as a professional. 


Principal focus or source of 
information being dealt with: 


The source of information is abstract 
or there and then, derived from 
readings, lecture inputs, compiled 
data, etc. 


Focus of information is on what and 
how to get task done, derived from 
previous work, plans, critiques, 
evaluations of progress, preparing for 
a presentation 


Nature of feedback and rcwar Is: 


Perfommnce is evaluated against right 
or best answer as judged by the body 
of knowledge or the icachar's expert 
opinion. 


Output is evaluated against criteria of 
practicality, feasibility, sellability, 
etc. 


Nature of learner's role: 


Activities and communications are 
governed by of inference, methods, 
terms, etc. often subject to learners 
memory recall. 


Learners make own decisions about 
use of rules* their time. Choice and 
actions at one point in time influence 
what occurs next. 


Teacher Role: 


Teacher is the expert authority, 
interpreting the field of knowledge or 
judging what is correct, competent, 
acceptable performance. Teacher may 
also be an enforcer of methods 
stipulated by the body of knowledge 
he represents. 


Teacher is a consultant or coach at the 
learners request in onJcr to advice or 
impart his knowledge of the field he 
represents. 


Insiructional Activities and 
Enrichment Components: 


Lectures, papers, printed materials, 
model building, projects, practice, 
presentations, feedback, analogies, 
repetition. 


Simulations, case study, laboratory, 
field work, projects, homework, 
group discussion, demos, structured 
activities. 


Leaminc Ability Best Supported: 


Abstract Concepiualizer 


Active Experimenter 
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Table 9. Means of Learning Style Dimensions Indicating Best Learning Ability with Most Appropriate Learning 



iVioiKl^vlIlull ollU 

Division Tvoe 


Learning Style Dimension of Learner and 
Most Appropriate Learning Environment 




AC-CE 
Mean 


Anorooriate 
Learning 
Environment 


AE-RO 
Mean 


Appropriate 

Learning 
Environment 


Upper Facilities 


-.09 


2) Affective 


7.00 


i)Behaviofal 


Upper Mid Facilities 


7.50 


1) Symbolic 


-3.42 


2) Perceptual 


Middle Facilities 


4.61 


1) Symbolic 


-.47 


2) Perceptual 


Supervisor Facilities 


3.87 


1) Symbolic 


-.36 


2) Perceptual 


Upper Probation 


5.42 


1) Symbolic 


-1.26 


2) Perceptual 


Supervisor Probation 


10.77 


1) Symbolic 


-3.50 


2) Perceptual 



environment 
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THE USE OF TELECOMMUNICATIONS TECHNOLOGIES IN LEARNING 
ENVIRONMENTS: PRESENT AND FUTURE 



Greg Bam Western Kentucky University 



Abstract 

For several years, telecommunications technologies 
have been used successfully in the delivery of 
instruction and in other educational environment 
activities. Leadership in the use of these technologies 
has often come from the state level. This paper will 
examine what some states are currently doing and 
explore some future models. 

Introduction 

There was a time not many years ago when the term 
"telecommunications" was used to refer to a technology 
that was used exclusively by the telq)hone company in 
the transmission of voice or character data as with the 
telegraph. The development of FNX technology 
expanded telecommunications to include the 
transmission of picture data over telephone lines. The 
creation of broadcast technologies broadened 
telecommunications to include media other than the 
twisted pair. The computer age further broadened the 
term's meaning to include the transmission of digital 
data between machines that wave in close proximity to 
each other or separated by large distances. Satellites 
have played a major role in the expansion of the 
meaning of telecommunications. 

Today, telecommunications and its related technologies 
play an increasingly significant role in the lives of all 
human beings. New applications and technologies are 
being continuously developed. These applications 
include the delivery of instruction; in school and 
training environments. 

Leadership in the use of telecommunications for 
learning applications has come from a number of 
sources in the public and private sector. In the public 
sector, significant lead^hip has often come at the state 
level. Some examples of state leadership include what 
has been done and is currently in progress in Kentucky 
and Indiana, to name but two. 

This paper will examine what these two states have 
accomplished in their educational environments through 
the use of telecommunication technologies. It will then 
present some possible future ways that these 
technologies can be used to provide more effective 
learning environments. 



educational levels: elementary, secondary, 
coUege/univ^ity, and adult continuing education. 

The Commonwealth of Kentucky is one of the more 
interesting and progressive states in the use of 
telecommunications. Kentucky has some areas of fairly 
dense population but it is basically a rural state. Many 
of the school systems in the state are organized on a 
plan based or a county-wide configuration; at that, many 
of the systems have less than 1000 students in grades 
K-12. Because of this student body size, many systems 
simply have not been able to provide the variety of 
curriculum offerings that would allow students to 
successfully compete for scholarships in higher 
education. Because of financial constraints, school 
systems could not hire the quantity or quality of 
teaching professionals needed. 

The state recognized that telecommunications could help 
relieve its educational crisis in 1966 when its legislature 
created the Kentucky Educational Television network 
(KET). The thrust of KET was aimed at the elementary 
school and provided "in-schooP programs in a number 
of subject areas. The original transmission technique 
was the use of microwave transmitters and telephone 
lines. Network television in the state was fed 
throughout the state in the same manner. 

KET went on the air for the first time in 1968 and has 
been there ever since. Transmitted from studios in 
Lexington, programming soon spread to the secondary 
level and, in the mid 1970s, to higher education and 
adult continuing education, llie transmission technique 
then moved to the use of microwave transmitters on the 
Kentucky Early Warning System (KEWS) as the carrier. 
To alleviate problems caused by the heavy demand for 
programming and the limitations of the capacity of the 
microwave/KEWS system, the 1988 state legislature 
authorized the creation of the Star Channel which could 
be broadcast via satellite. The same legislature also 
funded the purchase of a downlink dish for every school 
building in the state; the last of the dishes is to be 
installed by August 1990. 

A significant activity of KET today is the delivery of 
advanced placement courses such as calculus and 
physics. Thus, many more students are able to take 
these courses where the opportunity did not exist before. 
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Applications of Telecommunications In 
the States 

A number of states have made extensive use of 
telecommunications to improve their learning 
environments. These applications have come at all 



The creation of the Star Channel has freed up the 
KEWS backbone for greater use by higher education in 
the state. Several state universities (Kentucky, 
Louisville, and Wester Kentucky) have also added 
satellite uplink capabilities and anticipate increased 
programming with this medium. 
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A number of telecourscs are now being offered for 
undergraduate and graduate crr:aii by state universities 
and community colleges through KET. There is 
continuous demand for more offerings. At some 
installations on the KEWS network, students have the 
capability of "call back" to a live program in a 
teleconferencing mode format These capabilities will be 
extended to more sites across the state in the future. 

Western Kentucky University has been a leader in 
innovative delivery techniques. One of its telccourses 
was recently offered via KET at the same time the 
course was being taught live on campus. Westem, in 
cooperation with A T & T, has successfully tested a 
two-way video capability between its campus in 
Bowling Green and its campus site in Glasgow, forty 
miles away. The system uses video comparison 
techniques on a dedicated line between the two sites. 
The instructor is able to see and hear the students at the 
same time as being seen and heard by the students. At 
present, the system is not yet operational because of 
budget constraints within the university but has a hope 
for future funding. 

Like its sister institutions, the University of Louisville 
and the University of Kentucky, Westem has the 
capability of uplinking telecourscs not only to KET but 
to nearly any remote site in the world. Ongoing 
discussions on the feasibility of such a program 
continue to be held, but no defmite plans have been 
made other than for KET. But, the possibility exists. 

Westem is also studying the feasibility of creating 
video, audio and data linkage across its campus utilizing 
cable television and fiber optic technologies. Academic 
and entertainment programs could be accessed from 
many different offices and dormitory rooms. The 
communications network would also link various 
computer networks across campus together through the 
use of bridges and gates. 

The State of Indiana has taken a slightly different 
approach to the use of telecommunications. The 
impetus there has come from the institutions of higher 
education, primarily Indiana University, Purdue 
University and Ball State University with support at the 
state level, 

Perhaps the first use of te'ecommunications in the state 
came from the IMPATI project at Purdue University in 
the 1950s. This project transmitted television programs 
to elementary and secondary schools from a DC-3 
airplane which was airbome over thv*. central part of the 
state. Programming covered a variety of disciplines and 
had some success. Lack of funding eventually caused the 
project to be discontinued 

In the 1960s, the state funded two new projects, both of 
which exist and flourish today.^ The first was the 
creation of the SUVON (State University Voice 
Network). This project involved the use of microwave 
and telephone technologies that connected all public and 



private universities and colleges in the state into a state- 
wide voice communications network. Faculty and staff 
at individual institutions could communicate via a 
"local call" to other institutions of higher education in 
the state. In some locations, communications was 
possible to the local community in which the 
destination insutution was located with a simple use of 
a dial tone. In later years (and currently) the network has 
been used for teleconferencing. It has also been used as a 
medium for computer communications. For example, 
the seven campuses of Indiana University use the 
SUVON network to communicate to the computin/, 
facilities at the main campus (in Bloomington) and at 
the lUPUI campus in Indianapolis. 

The decade of the I960's also saw the creation of the 
Indiana Higher Education Telecommunications System 
(IHETS). The system was created for the purpose of 
offering college-level and adult continuing education 
courses in a variety of disciplines throughout the state. 
The system featured the use of one-way video with two- 
way audio capability. Location of sites on the IHETS 
network has grown far beyond university/college 
campuses. Sites are also now located in hospitals and 
county extension service offices. A number of certificate 
programs have been offered to practicing professionals 
and an expansion of this service is anticipated. 

To date, the IHETS network has functioned as a closed 
circuit network, that utilizes teleconferencing. 
Discussions are underway relative the use of cable and 
satellite for broader coverage. Satellite transmission is 
currently being used for some applications but is still 
closed circuit in nature. The use of two-way video is 
also being explored by IHETS. 

The IHETS network is beginning to include 
programming to some secondary schools in the state. 
Ball State is cunently providing this programming in 
several areas and it is expected that Indiana and Purdue 
will soon follow suit. Cunently, the state is exploring 
ways of expanding state-wide education to school 
districts. It is examining models such as the one in 
Kentucky but a concentrated effort at the state level is 
not expected momentarily. 

Kentucky and Indiana are but two examples of how 
telecommunications are being used on a state-wide basis 
to provide richer opportunities for citizens in the state. 
Different models have been used by the two states but 
interest is growing as the potential of 
telecommunications technologies continues to be 
recognized. In both cases, the primary barrier is cost. 
The effect of this barrier remains to be seen in each 
individual state. 



The Future 

What is the future for the delivery of instruction via 
telecommunications technologies on a wide scale basis? 
The cost of implementation of these technologies will 
decrease and become more affordable. Delivery 
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techniques will improve; the development of feasible 
two-way video will be a major factor. 

Let us speculate for a few moments on some possible 
scenarios. Students will be able to interact visually and 
orally with their teachers from remote sites* just as 
though in the actual classroom. Students in English 
composition courses will be able to transmit their work 
to their instructor electronically with immediate 
feedback and instruction* 



Computer science students will be able to intemct on a 
regular time basds with their instructors firom computer 
systems. More and more programs will be (rffeitd to 
students at upper educational levels as the potential of 
full two* way video is realized. Educational oppcxtunities 
will truly become available to everyone. 

Greg Baur 

Department of Computer Science 
Western Kentucky University 
Bowling Green^KY 42101 
(502)843-4273 
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AN INTERACTIVE TELECOMMUNICATION NETWORK TO SUPPORT EDUCATORS: A 
PRELIMINARY REPORT ON THE VEIN PROJECT 



Barbara T. Hakes and Landra L, Rezabek, The University of Wyoming 



Abstract 

The Video Education Interactive Network (VEIN) 
currently being established by the University of 
Wyoming, the Wyoming Goodlad School-University 
Partnership, and private business represents a 
collaborative effort to improve teacher preparation and 
practice through the use of telecommunication 
technologies. Goals, development concerns, and 
anticipated uses of the VEIN system are reviewed. 
Recommendations for similar collaborative projects 
designed to support preservice and inservice teacher 
education ait included 

introduction 

The nation has called for reform at all levels of public 
education, demanding that our new teachers receive more 
extensive prq)aiation in the content areas for which they 
will be responsible. It is common knowledge that good 
pedagogical practice must be acquired simultaneously 
with this content area expertise. In order to prevent new 
teachers from engaging in instructional malpractice for 
years before developing real teaching skills, pedagogy 
must be embedded in undergraduate education programs 
and a coaching or mentoring system must be available 
to preservice teachers which will also support them 
throughout their initial years as professionals. 

In the best of all possible worlds, this support could 
take the form of an experienced professional who would 
serve as a mentor/coach to student teachers. A 
mentor/coach would be readily available to student 
teachers and would be able to observe and interact with 
them frequently while pxeservice teachers were engaged 
in classroom activities with learners. Though 
observation and feedback is included as a standard 
component in the supervision of student teachers, 
cunent methods which rely on the presence of a live 
supervisor or on the use of videotaped lessons typically 
provide feedback after the student teacher has completed 
his or her interaction with students. Too often, post-hoc 
suggestions made to student teachers have little 
opportunity to be incorporated into practice or to be 
tested by preservice teachers in a supervised 
environment. 

An alternative approach to the live supervision and 
mentoring of student teachers being examined at the 
University of Wyoming will utilize the capabilities of 
telecommunications media to provide students with 
almost instantaneous feedback on their pedagogical 
practice. The scenario envisioned is not unlike the way 
in which information is exchanged between the referees, 
the players, and the instant replay booth in a 



professional football game or the the way in which 
diagnostic and intervention strategies are used by a 
surgical team in a modem hospital. A student teacher's 
classroom is a serious clinical laboratory setting where 
coaching, mentoring, and the exchange and management 
of visual and audio information must occur in order to 
produce outstanding texhers. A network which provides 
real-time voice and video exchange is required to provide 
the channel for such an interactive mentoring/coaching 
system for preservice teachers. Such a system has 
potential for a multitude of other teacher-training 
applications, particularly as a support for new teachers 
among whom the drop out rate within the profession is 
quite high. 

Efforts to incorporate telecommunication technologies 
in the support of student teachers (Johnson and 
Clawson, 1989) and beginning practitioners (Rodman, 
1989) suggest that supervision, mentoring, and 
coaching can be accomplished from a distance by 
coordinating the efforts of student teachers, cooperating 
professionals, and college faculty. In Wyoming, a state 
where vast geographic distances separate population 
areas, attractive incentives exist to test the potential of 
telecommunications technologies to support student 
teachers and beginning profc<!sionals. For example, the 
College of Education at the University of Wyoming 
actively is seeking alternative approaches to improve 
teacher education and is investigating the opportunities 
provided by telecommunications technologies to expand 
student-professor interaction across the state. Another 
incentive for testing the potential of technology to 
support teacher preparation is the fact that, as a member 
of the Goodlad School-University Partnership, the 
College works closely with public school districts in 
Wyoming to improve the quality of education within 
the state and across the nation. The Wyoming Goodlad 
Technology Task Force cunently is focusing on the 
potential of telecommunications to provide alternative 
educational opportunities. This existing working 
relationship among College faculty and public school 
administrators and teachers, along with their common 
interests in the potential uses of telecommunications to 
improve teaching and learning, led to a natural 
collaborative venture triggered in part by the 
opportunity of partial funding through a grant from 
private enterprise. To achieve reform in education, 
collaborative ventures between all levels of education, 
as well as business and government, are needed to create 
networks for the training and development of 
professionals and to identify sound methodologies for 
the incorporation of information technologies into our 
educational system. 
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Several factors contribute toward making the College of 
Education at the University of Wyoming a key location 
for a demonstration project of this nature. The 
University of Wyoming offers the only four year teacher 
training program in the Slate. Practicing professionals 
in the State are separated by vast geographic areas 
making it difflcult for them to network resources with 
the University. The College is in the process of 
restructuring its preservice teacher education, adopting a 
change-based model. Part of the change-based model 
involves teacher training in the use of new information 
technologies. The College has already established a 
strong human network of educators through the Goodlad 
SchoolAJniversity Partnership Program. This human 
network has produced substantial positive differences in 
the schools and in the University's Teacher Education 
Program. The University has committed itself to help 
expand and enrich the curriculum in Wyoming's public 
schools. The College plans to participate, at many 
levels, in some of the already existing and effective 
information exchange networks such as the Big Sky 
Telegraph system. These systems are making possible 
the exchange of information between our educational 
institutions, government agencies, agricultural 
extension network, and businesses in isolated 
communities. Our teachers and children must be 
educated to utilize such systems if they are to be 
prepared to serve our increasingly information oriented 
society in productive ways. The availability of 
information networks and people who are 
knowledgeable about their uses also provides attractive 
incentives for businesses to locate in areas where these 
resources are available. 

The College of Education, with its Laboratory School 
and its commitment to the Goodlad Project provides an 
ideal setting in which to create a demonstration 
program. The program can be used as a resource to 
attract additional funding to involve nationally 
recognized scholars in educational research and grant 
writing, and to entice other institutions to engage in 
collaborative teaching and research efforts. The oroposed 
Video Education Interactive Network (VEIN), should 
serve as a continually expanding resource which will 
become self supporting through its ability to attract 
new sources of funding. It will become a laboratory for 
the integration of new technologies in preservice 
experiences, teaching, learning, research, faculty 
development, and inservice education. This paper 
represents the preliminary summary of the Video 
Education Interactive Network (VEIN), a collaborative 
demonstration project for the improved support of 
preservice and beginning teachers. 

Project Goals 

The initial focus of this project is the collaborative 
development of a real-time two-way voice and video 
network to improve teacher education. The VEIN 
Project first will target the development of a mentoring 
and coaching system for student teachers. Specific goals 
include but are not limited to the following outcomes. 



When compared with student teachers participating in 
the traditional student teaching program at the 
University of Wyoming, student teachers participating 
in tlie VEIN program will: 

1 be more adept at teacher decision making. 

2. be more highly skilled at diagnosing individual 
student dilemmas. 

3. demonstrate enhanced communication skills. 

4. be better able to diagnose inavidual learning 
style preferences. 

5. be more likely to use mentoring/coaching 
techniques with their own students. 

6. be more likely to incorporate the use of 
technologies in their own classrooms. 

After the first phase the VEIN Project has been 
evaluated and revised, expansion of the network will 
involve the creation of a teacher support system for 
first- and second-year teachers and eventually the VEIN 
Project will evolve to include in-service support for 
more experienced educators. An additional long-range 
goal of the VEIN Project is the development, 
dissemination, and evaluation of instructional strategies 
with which classroom teachers should be familiar in 
order to appropriately integrate telecommunication 
technologies into their classrooms. It is critical that the 
instructional applications of new technologies be 
incorporated into teacher education programs so that 
teachers learn how to use technologies to improve 
educational experiences for our nation's youth. 

The VEIN Project is being continuously evaluated on 
its effectiveness in providing a teacher support network 
and its potential for expansion to meet other needs. As 
it evolves and develops, the VEIN Project will be 
expanded to test its potential for enhancing educational 
practice in the following ways: 

1. Create a much stronger educational system at 
all levels within the state by providing a 
means for the networking of a range of 
resources. 

2. Provide preservice and practicing teachers with 
an opportunity to gain expertise with the 
integration of telecommunications media as 
part of the teaching/learning process. 

3. Provide learners with an opportunity to gain 
experience with telecommunications media as 
part of the teaching/learning process. 

4. Provide opportunities for students at 
geographically separated schools to engage in 
cooperative learning activities, 

5. Impact the integration of technology into 
classrooms in other states given the fact that 
70% of the College's students leave to go to 
41 other states. 

6. Facilitate cooperative teaching and learning 
programs between the College and school 
districts around the State and the Nation. 

7. Provide new opportunities for faculty 
development and in-service experiences and 
allow for interaction between teachers who 
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might otherwise have a very limited peer 
group. 

8. Facilitate one-on-one delivery for the 
developmentally disabled. 

9. Help expand and enrich the curriculum in 
Wyoming's public schools by delivering 
content from the University to districts which 
can not afford to hire teachers for certain 
subject areas. Advanced students in Wyoming's 
tiigh schools could simultaneously take 
courses with college students at the University. 

10. Enhance collaboration between the Colorado 
Goodlad School-University Partnership and the 
Wyoming Goodlad School-University 
Partnership. The laboratory schools at the 
University of Wyoming and Northern Colorado 
University could become a national laboratory 
school for other teacher education programs. 

1 1. Allow the Goodlad Partnership concept to be 
expanded to include the private sector. 

Development and implementation Issues 

Several factors have contributed to the collaborative 
development of the VEIN P.^oject. Practicing 
professionals in Wyoming are separated by vast 
geographic areas making it difficult to network 
resources. However, the Wyoming Goodlad School- 
University Partnership has promoted interaction among 
the College of Education and school districts across the 
state. The University and the participating Goodlad 
districts are committed to expanding and enriching the 
curriculum in Wyoming's public schools. Part of this 
commitment targets improving experiences for 
preservice teachers and includes incorporating the uses 
of instructional technologies into teacher training 
programs and classroom practice. The cooperative 
netwoiic and common goals already established among 
the Goodlad School-University participants facilitated 
the development of the VEIN Project 

The College of Education, the University of Wyoming, 
and the Goodlad school districts were prepared to make 
fiscal and programmatic commitments to improving 
student teaching experiences through the use of 
telecommunications. The financial support provided by 
private business demonstrates the interrelated concerns 
shared by public schools, higher* education, and business 
in developing the role which instructional technologies 
play within the educational system. Programmatic 
commitment to the VEIN Project is demonstrated by 
the willingness on the pan of the College and school 
districts to work together to develop distanced 
mentoring/coaching/supcrvisory experiences for student 
teachers. These particular concerns are elaborated below. 

Required Technological Resources. Each 
participating institution must have access to a video 
compression unit ranging in cost from $18,000 to 
S30,000, depending upon the quality of video desired. 
Additionally, each participating institution must be 
prepared to share the costs of leasing either a fiber 
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optics or Tl land line for transmission of the 
compressed video and audio information. Because of the 
relatively expensive cost ($6,000 to Sl2,000 per month 
depending upon the quality of video signal required) 
collaborative efforts between public institutions and the 
private sector are extremely advantageous. The VEIN 
Project is based upon this type of collaboration. 

Coordinating Student Teaching/Mentoring 
Experiences. A major issue involves the 
coord' laiion of students, faculty, master teachers and 
school districts. Under the present live visitation model 
being used at the University of Wyoming, site visits are 
made during the student teaching experience by College 
field-based faculty who coordinate their visitations with 
the master teachers in a school district The VEIN 
Project supports the direct involvement of students on 
campus in Laramie with master teachers in eight 
disuicts around the State prior to student teaching. 
These students will team teach with master teachers 
using the two-way audio and video network. Topics for 
team teaching are collaboratively identified by methods 
classes on campus and participating teachers a*, the 
network sites. Students identify a topic and consult with 
the practicing professional associated with that topic. 
Once the lesson is prepared students on campus use the 
VEIN network to team leach the lesson with their 
cooperating teacher. 

Technology Training. Telecommunications 
training for faculty, students, and muster teachers is 
important in the sense that they must be familiar 
enough with the technologies involved to make the 
systems being used seem almost uansparent That is, 
the technology should interfere as little as possible with 
the accomplishment of the insuuctional goals. An 
interesting lesson was learned during a demonstration of 
the compressed video portion of the network on August 
2 and 3,1990. / rehearsal with the equipment was held 
on Aug ust 2, and two one-hour demonstrations were 
conducted on August 3 for the University of Wyoming 
Board of Trustees' Symposium. By the second 
demonstration, the grade-school students, classroom 
teachers, student teachers, and faculty members involved 
in the demonstration were almost completely at ease 
with the system. The most difficult feat to accomplish 
was to dispel the mindset that the system could only 
send QL receive information. Once participants found 
they could interrupt each other and engage in uue 
interaction, the technology became nearly transparent 
after only a few hours of experience with the system. It 
became quite apparent, however, that technical support 
be provided so that the users do not have to worry about 
operating the system they are using. 

Trouble Shooting and Technical Support. A 
discussion of technical support leads to issues 
concerning the trouble shooting of the technology If a 
technician is available to users, the equipment can be 
maintained and the system adjusted by the technician 
should operating difficulties occur. Providing technical 
assistance helps in trouble shooting if-any glitches 
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occur, but also helps encourage the use of technologies. 
If people have a high level of anxiety abou* using the 
equipment, they are less likely to use the system. If 
people do use the system, i is likely they wil! not be 
relaxed enough with its use to gain the full benefits 
possible, and technical support lessens these concerns. 
Experience with the VEIN demonstration sessions 
indicates that money spent to provide technical support 
for the system is a wise investment. 

Evaluation and Educational Importance 

The development of a telecommunication-based 
mentoring/coaching system for student teachers is the 
first goal of the VEIN Project The project will undergo 
continuous formative evaluation and subsequent 
refinement in order to ma^vimize its effectiveness and 
efficiency for all participants involved in the 
collaborative effort An advisory committee has been 
formed consisting of representatives from the public 
schools, community colleges, the University of 
Wyoming, and the private sector to assess the degree to 
which the objectives of the VEIN Project outlined 
above are being accomplished. Recent criticism of 
teacher education has included recommendations for 
additional research on increased experience-based or 
clinical experiences prior to student teaching. The level 
and significance of research on the effectiveness of 
increased clinical experiences earlier in teacher 
preparation programs is limited. The VEIN Project 
provides an opportunity to study numerous issues 
related to the improvement of teacher education and 
practice in the schools. A range of qualitative and 
quantitative research methodologies and research designs 
can be supported through the VEIN system. The results 
of such research may be more directly applied to the 
improvement of schools and teacher preparation. 

Summary 

The initial focus ot the VEIN Project is the 
collaborative development of a real-time two-way voice 
and video network to improve teacher education. The 
VEIN Project first is first targeting the development of 
a mentoring and coaching system for student teachers. 
After the first phase of the VEIN Project has been 
evaluated, expansion of the network will involve the 
creation of a teacher support system for first- and 
second-year teachers and eventually the VEIN Project 
will evolve to include in-service support for more 
experienced educators. An additional long-range goal of 
the VEIN Project is the development, dissemination, 
and evaluation of instructional strategies with which 
classroom teachers should be familiar in order to 
appropriately integrate telecommunication technologies 



into their classrooms. It is critical that the instructional 
applications of new technologies be incorporated into 
teacher education programs so that teachers !eam how to 
use technologies to improve educational experiences for 
our nation's youth. 

Several factors have conuibuted to the collaborative 
development of the VEIN Project. Practicing 
professionals in Wyoming are separated by vast 
geographic areas making it difficult to network 
resources. However, the Wyoming Goodlad School- 
University Partnership has promoted interaction among 
the College of Education and school districts across the 
state. The University and the participating Goodlad 
districts are committed to expanding and enriching the 
curriculum in Wyoming's public schools. Part of this 
commitment targets improving experiences for 
preservice teachers and includes incorporating the uses 
of instructional technologies into teacher training 
programs and classroom practice. The cooperative 
network and common goals already established among 
the Goodlad School-University participants facilitated 
the development of the VEIN FtojecL 

The College of Education, the University of Wyoming, 
and the Goodlad school districts were prepared to make 
fiscal and programmatic commitments to improving 
student teaching experiences through the use of 
telecommunications. The financial suppoit provided by 
private business demonstrates the interrelated concerns 
shared by public schools, higher education, and business 
in developing the role which instructional technologies 
play within the educational system. Based on 
demonstrations already conducted, technical support, 
practice with the system, and hands-on use by practicing 
educators, student teachers, faculty members will 
contribute to the success of the VEIN Project This 
commitment to the VEIN Project is demonstrated by 
'he willingness on the part of the College and school 
districts to work together to develop distanced 
mentoring/coaching/supcrvisory experiences for student 
teachers. 
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TEACHING TELECOMMUNICATIONS: 
PROMOTING PRACTICE AND POTENTIAL 
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Abstract 

Issues concerning the integration of telecommunications 
technologies into educational practice are addressed from 
the perspective of students, instructors, and seminar 
participants who were involved in a course in 
Instructional Telecommunications, Suggestions for the 
design and implementation of similar college courses 
include the use of practical, handsH)n activities, an 
emphasis on design and development aspects of using 
telecommunications systems for instruction, and 
attention to principles of change agentry. These 
guidelines also ^pear to be impropriate considerations 
for those people interested in promoting the use of 
telecommunications media in classrooms and other 
instructional situations. 

Introduction 

The term telecommunications is filtering more 
frequently into the conversations of educators. Though 
some observers (Hudspeth and Brey, 1986) point out 
that inertia tends to slow the adoption of 
teleconununication media within school systems, they 
also stress the potential conuibutions which 
telecommunication technologies can make to the 
instructional process. Others (Roberts, Blakeslee, 
Brown, and Lenk, 1990) offer a more positive 
assessment of the rate at which telecommunications 
media are affecting educational practice and underscore 
the need for professional educators to become familiar 
with the instructional capabilities of telecommunication 
systems. 

Though current uends reflect an awareness of the 
potential of telecommunications to affect educational 
practice, a myriad of unanswered questions remain 
concerning the "whys" and "how-to's" of integrating 
telecommunications into education. In addition, the 
defmition of telecommunications itself is unclear, as 
technological developments offer an increasing variety 
of delivery media and enable users to combine elements 
of telecommunication systems in unique ways. If 
educators are to take advantage of these new 
telecommunication technologies, they must not only 
become aware of the potential uses of the technologies 
and acquire basic operational knowledge of the 
telecommunication systems, but they must also address 
associated instructional design issues, be familiar with 
system limitation and strengths, determine instructional 
strategies, and focus on appropriate learner outcomes 
that will be facilitated by the incorporation of 
telecommunications into the classroom. 



Though telecommunication systems no doubt are 
appropriate for instructional situations outside of the 
Uaditional K-12 context, a primary mission of the 
College of Education at the University of Wyoming is 
to prepare teachers to assume responsibilities in the 
public schools at these levels. Toward this end, a 
graduate course in instructional telecommunications was 
designed to help prepare teachers and other educators to 
incorporate telecommunications into their instructional 
situations and to help their colleagues do likewise. This 
paper deals with development concerns associated with 
the telecommunications course; relates successes and 
frustrations encountered during the semester, offers 
recommendations for the design and delivery of similar 
courses; and enumerates suggestions from students, 
instructors, and other educators for integrating the use of 
telecommunications in public schools and higher 
education. 

Course Description and Course 
Development Concerns 

The Instructional Telecommunications course taught 
during the Spring Semester of 1990 was designed to 
familiarize students with a variety of current 
instructional telecommunications systems. The 
following concerns were discussed for each of the 
systems studied in class: 1) system capabilities, 2) 
hardware and software requirements, 3) costs, 4) 
management requirements, S) instructional advantages 
and disadvantages, 6) instructional strategies and skills 
appropriate for each system, and 7) implications for 
instructional design. The telecommunications 
technologies targeted in this course included: 1) satellite 
and microwave. 2) audio and audiographic, 3) 
compressed video, 4) computer-based networics, and S) 
closed circuit and cable. A primary goal was to design a 
course that was hands-on and that would allow educators 
to gain experience with telecommunications systems 
actually being used in the state. The selection of the 
five systems mentioned above reflects the use of 
technologies actually available for hands-on use at the 
University of Wyoming and in some of the K-12 
schools across the state. Hie integration of these and 
other technologies to form additional "hybrid** 
telecommunication systems also was discussed during 
the course. 

During the semester, students investigated each of the 
Ave types of telecommunications systems enumerated 
above in relation to each of the seven concerns listed. A 
goal of the course was for students to gain basic 
familiarity with a variety of telecommunications 
systems. An additional goal was for students to 
demonstrate basic operation of two of the 
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telecommunication systems as part of a group project. 
Successful completion of the group project included the 
development of seminars dealing with instructional 
applications of the telecommunications technologies 
with which the student groups had become familiar. 
These group seminars were pre'^entcd to College of 
Education faculty and students, and the group members 
actually used the technologies being discussed to present 
the content of the seminar. Therefore, group members 
not only discussed instructional issues concerning each 
of the telecommunications systems, but also 
demonstrated appropriate design and delivery strategies 
associated with each technology. As a result, students 
enrolled \n}hc Instructional Telecommunications course 
served as role models and change advocates for their 
colleagues in the College as they demonstrated the 
instructional potential of the telecommunication 
systems. 

The student*delivered seminars were includc<I in the 
course for a dual purpose. The first was to help students 
realize the critical nature of the instructional design 
process when using telecommunications technologies 
for educational purposes. Too frequently, students focus 
on the operation of the hardware itself and not on the 
ways in which technologies can be used to enhance 
instruction. The seminars encouraged students to think 
about important instructional attributes of the 
technologies and to share this information with others 
in the College. These seminars also helped accomplish 
another goal which was to make telecommunications 
technologies more visible within the College and to 
stimulate the interest of faculty and students in the 
instrucuonal potential of these delivery systems. The 
seminars were videotaped and will be available to 
faculty and students for independent viewing and for 
faculty development 

Additional class acti\ ities required students to analyze 
instructional situations, to specify what types of 
telecommunications systems they would incorporate to 
meet the instructional needs described, and to justify 
their choices of delivery systems. In doing so, students 
discussed current research and practice associated with 
instructional telecommunications systems. Required 
readings included Instructional Telecommunications: 
Principles and Applications (Hudspeth and Brey, 1986) 
and numerous readings selected by students themselves. 
In the process of preparing the group seminars about 
specific systems, each group of students assembled key 
readings on that particular technology and these 
resources were shared with the other members of the 
class. In addition to a variety of readings, lectures, and 
seminar presentations, instructional activities also 
inch J numerous site visitations, hands*on sessions 
with hardware and software, and discussions with guest 
speakers, many of whic*i were conducted using audio 
teleconferencing technologies. Again, course activities 
were designed to provide students with opportunities to 
become aware of a variety of telecommunicatfcns 
technologies and to encourage students to think about 



the potential of using these delivery systems to provide 
richer educational environments. 

Successes 

A major goal for the course was to provide students 
with opportunities to become familiar with the hardware 
aspects of telecommunications systems as well as with 
the particular design and development issues which 
facilitate the instructional uses of the technologies. 
Though the hands*on*the-hardware approach utilized in 
the course attracted many students at the onset, students 
also rapidly realized the importance of learning not only 
how to operate the equipment but of knowing how to 
plan for and carry out instructional telecommunication 
sessions. Students demonstrated the tise of technologies 
and also nKxIeled appropriate ways to address design and 
delivery issues associated with the use of 
telecommunications. The importance of maintaining 
this balanced approach to incorporating 
telecommunication systems into educational 
environments proved to be one of the greatest '*truths'* 
which emerged during the semester. From the 
perq)ective of both the students and the instructors, one 
of the successes of the course was that students realized 
the critical importance of the instructional design 
process when telecommunication technologies are used 
to deliver instruction. 

The course was also successful in increasing student 
awareness of and enthusiasm for the use of 
telecommunication technologies in the educational 
environment Student evaluations of the course showed 
that most class participants had begun the semester with 
little or no knowledge of instructional 
telecommunications. Most students also admitted that 
prior to the course they had been intimidated by the 
technologies themselves. Student feedback indicated that 
class participants uMt more comfortable with and more 
knowledgeable about instruct' )nal telecommunication 
technologies by itie end of the semester. In fact, the 
course seemed to have a residual effect on several 
students. As one student commented several months 
after the class, she "didn't realize it then** but she had 
learned enough during the course to be able to make 
significant contributions to a discussion about 
telecommunications that had evolved during an adult 
education course that the student took the following 
semester. Another student confided that, months after 
the course, she had felt confident about her responses to 
questions regarding telecommunication technologies 
that had been posed during her successful interview for a 
position as an elementary library media professional. 
Course evaluations and student testimonials indicate 
that the course was successful in lessening student 
anxieties about telecommunication technologies and in 
increasing student knowledge and enthusiasm. 

Another successful aspect of the course was the hands- 
on nature of the class. Students valued the opportunity 
to gain practical experience using a variety of 
technologies. By preparing and delivering group seminar 
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presentations, students encountered flrst-hand many of 
the difficulties in using telecommunication 
technologies. Of particular interest were student 
comments that reflected their understanding that not 
only facilities and technologies are necessary to support 
instructional telecommunication endeavors, but that 
human resources arc a critical factor as well. Instructors 
and students had discussed this issue in class, bat the 
importance of human resources became much more 
apparent when students began a'^tuai hands-on 
development and delivery activities using the 
technologies. Instructors and students were fonunate to 
have the support and cooperation of many key people 
both inside and outside the College of Education as well 
as outside the University of Wyoming. As students 
soon learned, these human resources were csy^ntial to 
the success of their telecommunication endeavors and to 
the overall success of the course. 

The group seminar piescntations provided students with 
hands-on expenence using telecommunication 
technologies from the perspective of an instructor or 
instructional designer, but the course also provided class 
participants with an opportunity to experience what 
**real students** would encounter during irstruction 
delivered using a variety of telecommunications 
technologies. When seminar sessions were presented, 
the remaining class members experienced each of the 
delivery systems from a learner's perspective. In 
addition, audio teleconferences were conducted with 
DeLayne Hudspeth, author or the textbook, and Frank 
Odasz, director of Big Sky Telegraph, which provided 
students with an example of putting ''theory** into 
practice* Telecommunication technology also was used 
to solve an unanticipated real-world problem that arose 
during the course. When the time scheduled for the oral 
portion of the final examination for the class arrived, 
one instructor and one of the students were ill, and one 
student was out of town for a job interview. The oral 
portion of the flnal exam was conducted using audio 
teleconferencing, enabling all students to participate in 
the final. Students commented that this activity was an 
appropriate conclusion to the semester. Overall, the 
practical, hands-on nature of the course contributed to 
its success in the variety of ways discussed above. 
Limitations of the course and areas in which the course 
can be improved are discussed below. 

Recommendations and Caveats 

The hands-on nature of the course was one of the 
greatest successes of the class, but providing this type 
of experience generated some concerns. One student 
lamented the lack of formal lectures and paper-and*pencil 
tests; however, the course instructors believe the 
majority of students appreciated the break from the 
"traditional** graduate course format. To succeed in the 
Instructional Telecon:munications course, suidents had 
to become active participants. Students could not sit 
passively, take notes and regurgitate information on a 
tesL Instead, they had to locate resources, work in 
groups, plan seminars, and learn to operate hardware. 
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Many students expressed an interest in having more 
hands-on experiences, particularly with the technologies 
other than those which they had studied in depth in 
preparation for the seminar presentations. Other students 
commented on both the benefits as well as the 
frustrations of working in groups. Benefits included 
sharing responsibilities and resources, while frustrations 
arose in scheduling meetings and in a ligning 
responsibilities. Most of the students felt hat the 
preparation of groups scjninars was worth the minor 
frustrations encountered, particularly because the 
seminars offered the students an opportunity to make a 
real-worid conuibulion to faculty and students within 
the College of Education. Though students were 
nervous prior to their seminar presentations, students 
commented that a sense of achievement and self-esteem 
resulted from the experience, tudents also were pleased 
that faculty members and fellow students had informally 
thanked them for organizing the presentations and had 
encouraged more seminars of a similar nature. Based on 
student feedback, when the Instructional 
Telecommunications class is offered in the future, the 
instructors again will develop a hands-on course and 
will more vehemently advise students that the course 
demands active participation. Instructors also will try to 
provide more time for students to individually use the 
variety of delivery systems available. Although 
expectations for group work and group process!ng 
strategies were discussed briefly in class, the instructors 
may also include more emphasis in this area. Finally, 
examples of frustrations and successes which students 
experienced in preparing and delivering the group 
seminars as well as feedback from seminar participants 
will be available to share with futui^ students. 

Feedback from the faculty and students who attended the 
seminars was e .tremely supportive of the sessions. 
Paiticipants appreciated the real-world demonstrations of 
system capabilities. Participants would have liked to 
have had more time to manipulate the technologies 
themselves with the help of the presenters, and this 
recommendation will be passed along to students in 
future Instructional Telecommunications courses. 
Seminar participants also commented that the audience 
consisted of people with a diverse amount of 
technological experience and suggested that seminars be 
o^ered on beginning, intermediate, and advanced levels. 
These comments both frustrated and encouraged the 
students in the class, who had debated audience analysis 
issues, had weighed their group resources and 
constraints, and had chosen to present information for 
the novice and to advertise the seminars as such. 
Students in the course felt that time, resources, and 
audience analysis had indicated that novice sessions 
would be most appropriate, but they were pleased to 
hear that seminar participants were interested in 
additional and higher level presentations. In fact, 
numerous seminar participants suggested that all faculty 
members and students in the College would benefit 
from similar seminars. These statements encouraged the 
students, for, though the people who attended that 
seminars were enthusiastic, students who* had prepared 
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the seminars *had been disappointed initially in the 
number of people attending which ranged between 
twenty and forty individuals per session. Course 
discussion regarding the process of educational change 
evolved naturally from these observations. Overall, 
participant feedback indicated that future students should 
continue the practical, demonstration-oriented 
perspective of the seminars but should attempt to 
provide participants with mcH'e hands-on experience with 
the technologies and to accommodate participants with 
varying levels of expertise. 

Students in the course commented that they did not 
know how to deal with the participant recommendations 
that seminars be targeted for individuals of varying 
levels. Student groups had made conscious decisions to 
design seminars for a general, novice audience and were 
frustrated by the amount of information that had to be 
omiucd from the seminars due to the one-hour format 
The course instructors encountered similar frustrations 
when designing the class itself. Though the majority of 
students were unfamiliar with the variety of 
telecommunications hardware, ind' ^ual students were 
iuite familiar with a single technoiugy. The instructors 
encouraged students already familiar with one of the 
technologies to participate in seminar groups involving 
delivery systems with which they were not well 
acquainted. Some student expertise was not tapped due 
to this request, but individual students expanded their 
experiences with the variety of technologies. Another 
frustration from the instructors' point of view was 
determining the focus of the course, for a class in 
instructional telecommunications could include d variety 
of topics. This course focused particularly on selected 
delivery systems and the associated instructional design 
and delivery considerations, skirting many associated 
issues related to distance education. Students included 
some of these related issues during seminar 
presentations, and class discussions related to the 
Hudspeth text briefly touched upon some of the more 
global aspects of instructional telecommunications. 
However, another entire course is being planned to 
focus on additional principles, applications, and 
examples of distance education and the related 
technologies. The perspective of the additional course 
will include issues addressed in texts such as Readings 
in Distance Educations, Numbers 1 and 2 (Moore & 
Clark, 1989) and Linking for Learning (Office of 
Technology Assessment, 1989). The inclusive nature of 
the field of instructional telecommunications also 
provided the instructors w»th a challenge in the choice 
of textbooks. The Hudspeth text provided basic 
instructional design concepts and discussed 
administrative concerns with little emphasis on 
hardware issues, and students consulted current journals 
to familiarize themselves with the capabilities of the 
variety of hardware available. However, both instructors 
and students are still looking for an introductory text 
which consolidates information and issues in 
instructional telecommunications in one volume. 
Overall, instructors believe that the Instructional 
Technology course provided students with a basic 



understanding of what a variety of telecommunication 
technologies can do and of the design and delivery 
concerns associated with them so that students will be 
able to further explore the role of technology in distance 
learning in subsequent classes and in their educational 
environments. 

Opportunities for Integration 

During the semester, students in the Instructional 
TelecommunicaiU>ns class were asked particularly to be 
aware of ways to integrate telecommunications into 
classroom practice. Advantages and limitations of extant 
••packaged programs" such as TI-IN. Big Sky Tclegr^h, 
and CNN Newsroom were discussed. Students also 
investigated what educators needed to know in order to 
develop their own lessons and programs incorporating 
telecommunications technologies. The group seminar 
presentations provided students with an c^>portunity to 
••practice" promoting the use of telecommunications in 
one particular iucational environment, namely the 
College of Education. In this •'test case,** students soon 
began to see the value of being familiar with strategies 
to promote planned change (Havelock, 1973; Rogers, 
1982). For example, when presenting their group 
seminar sessions to faculty members and students 
within the College, students in the course observed 
typical trends in the adoption of innovations. Students 
noted that some of the individuals in the College 
demonstrated characteristics of innovators, early 
adopters, early majority, late majority, and laggards as 
described in Rogers' (1982) research on categories of 
those who adopt innovations such as 
telecommunications technologies. Also of primary 
interest to students in the class were tiiO seminar 
participants' comments which indicated that perceived 
relative advantage, trialability, observability, lack of 
complexity, and compatibility were indeed 
characteristics (Rogers, 1982) that would inaease the 
likelihood of the adoption of telecommunications 
technologies. For example, many seminar participants 
wanted to know how instructional telecommunications 
technologies would make their jobs as teachers easier or 
how they could teach better using the hardware 
(perceived relative advantage). Participants wanted to 
have opportunities to use the technologies themselves 
(trialability) and also wanted to see the various delivery 
systems working, particularly in situations similar to 
their own (observability). Seminar participants, perhaps 
initially overwhelmed by the less transparent delivery 
systems, felt that the telecommunications technologies 
were less complicated after seeing a dcmonsu^tion and 
receiving some some hands-on experience (lack of 
complexity). And many session participants appreciated 
the emphasis on how the use of technologies could be 
incorporated into existing classroom practice 
(compatibility). From their seminar experiences, 
students in the Instructional Telecommunications course 
began to understand how efforts to promote the u.se of 
instructional telecommunications technologies could be 
planned and delivered taking advantage of principles and 
practices of planned change. 
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Class participants also began to focus speciflcally on 
the potential for introducing and establishing 
telecommunication technologies in their own 
instructional siuiaiions. Two members of the class were 
library media professionals and were interested in 
establishing computer-based information links in their 
school media centers. Other class members interested in 
corporate training focused on the potential of interactive 
audio and video technologies in business. The 
classroom teachers enrolled in the course were 
particularly interested in establishing computer-based 
networks in their schools, and wanted to explore the 
opportunities to provide students with additional courses 
and with access to information outside of the school 
distinct using interactive audio and video delivery 
systems. Each student developed areas of concern, but 
all students were impressed with the need to educate and 
excite colleagues about the potential of instructional 
telecommunications. As mentioned above, both the 
seminar participants and the suidents in the instrucuonal 
technology course commented that the topic of 
telecommunications should be included in both graduate 
and undergraduate teacher education programs. Students 
in the course also advocated the use of 
telecommunications in the public and private schools. 
The common perception that instructional 
telecommunications will play an increasing rcle in 
education and that teachers should have both trainmg in 
and access to these resources was not paralleled by a 
shared perspective of how this goal should be 
accomplished. Neither did students find definitive 
answers ?n the literature. As students in the 
Instructional Telecommunications course learned, the 
potential of instructional telecommunications to impact 
students, teachers, and cihex members of the educational 
system appears promising, but the means towaid this 
end are still being investigated. Students found the 
principles of change agentry to be valuable tools for 
integrating telecommunications technologies into 
education. Students also observed that demonstrating the 
uses of telecommunications technologies and allowing 
potential users to gain some hands-on experience with 
the hardware lessened anxieties about using distance 
delivery systems for instruction. Based on experiences 
in the Instructional Telecommunications class, 
instructors also have begun to work with faculty at 
other iiistitutions to develop an inJra-university course 
in which students at geographically separate locations 
will investigate issues in telecommunications using the 
various delivery systems. Although much remains to be 
learned about the process for integrating 
telecommunications technologies into school systems, 
students in the Instructional Technology course believe 
telecommunications should be a priority item on the 
educational agenda and plan to incorporate some of the 
experiences and observations discussed above into their 
own instructional environments. 

Summary 

The potential for telecommunications to impact 
educational practice is a topic of current interest, and 
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examples of how telecommunication systems support 
instruction are becoming increasingly visible. In order 
to integrate telecommunications technologies into 
classroom practice, teachers must be provided with 
encouragement to adopt new instructional approaches, 
assistance in utilizing new technologies, and support in 
planning, implementing, and evduating insu-uction 
which incorxx)rates telecommunication technologies. 

In offering a hands-on course in Instructional 
Telecommunications for educators, it became apparent 
that efforts to integrate telecommunications 
technologies into classrooms will involve affective 
learning outcomes as well as focusing on the cognitive 
and psychomotor aspects of using the systems 
themselves. As students in the course observed, 
teaching educators how to operate the hardware and how 
to prepare and deliver effective lessons that will take 
advantage of the capabilities of distance delivery 
systems can only be accomplished after teachers have 
become aware of the potential of the technologies. 
Designing and presenting seminar sessions for members 
of the College of Education allowed students in the 
course to experience some of the rewards and 
frustrations often faced by those in the Held of 
instructional telecommunications. Students learned that 
several strategies known to contribute to the adoption of 
innovations can be used ic influence educators' attitudes 
toward the viability of telecommunications in the 
classroom and should be kept in mind when designing 
and implementing telecommunications training 
sessions. Students also found that emphasizing the 
design and development issues associated v*ith the use 
of telecommunications technologies were of paramount 
importance in the preparation of u^eir seminars, and 
design and development issues were of great interest to 
seminar participants as well. The use of demonstrations 
and hands-on activities also conuibuted to the success of 
the seminars. Similarly, both students and instructors of 
the Instructional Technology course felt that the 
emphasis on change agentry, instructional design and 
development, and practical experience were factors that 
contributed to the class itself. Though continuing 
modification of the cou.:>e vill occur, students, 
instructors, and seminar participants indicated that both 
the course and the seminars were valuable steps in the 
process of increasing educators* integration of 
telecommunications technologies into instructional 
practice. 
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A HYPERTEXT APPROACH TO STRUCTURING COMPUTER-MEDIATED SEMINARS 

AND CASE STUDIES 



Alexander Romiszowsid, Karen Lee Jost, and E-Cheol Chang, Syracuse University 



Abstract 

This paper reviews earlier experiences with the use of e- 
mail as a simple basis for computer-mediated seminars. 
Several approaches lo the construction of tools that 
assist participants in maintaining a structured view of 
the seminar over time are described. A pilot project that 
compares some of these tools is reported. 

Instruction Versus Conversation: 
Two Contrasting Paradigms 

We will not go deeply into the roots of the contrast 
implied by the title of this section. Suffice it to say that 
the main 'opposition camp* to the 'educational 
technology movement', since the days of programmed 
insuiiction and up to the present CAI systems, can be 
summarized by this title. A glance at Table 1 will 
explain what we have in mind. The instructional 
technology approach has been criticized on the grounds 
of; insisting on too specific, product-based and uniform 
objectives, and placing loo much emphasis on prior 
design and development of materials by the 'system' 
followed by the dissemination of these standard 
messages to all students indiscriminately. It has also 
been criticized for overemphasizing behaviors mastered 
rather than ideas processed, for the correction of errors 
rather than reflection on the implications of viewpoints, 
and for being shallow and superficial rather than 



encouraging the processing of complex multifaceted 
content 

Whereas this cannot be levelled as an across-the-board 
criticism of the educational technology movement (not 
all work in our field has followed the 'instructional' 
paradigm, nor is this paradigm always inferior to the 
alternatives), it has tended to limit our field to those 
who did not have a total aversion to the concept of 
'instruction' as defined in Table 1. 

Today, with the communication systems available, 
teleconferencing has become a widely used, versatile and 
ingenious methodology. In the State of New York 
alone, more than 100 conventional educational 
establishments use some form of teleconferencing to 
supplement, or supplant, conventional face-to-face 
education. Few of these would become so deeply 
involved with self-instruction-based education. Part of 
the reason may be the clash of paradigms, mentioned 
earlier. But perhaps a more compelling set of reasons 
springs from practical and economic considerations. 
ArmsLt)ng (1990), Brockley (1990), and Chute (1990), 
show how in vastly different contexts, it is both 
relatively easy and relatively cost-effective to set up a 
teleconferencing-based distance education system. 
George (1990) argues that the use of teleconferencing is 
only slightly different from the small group learning 
activities that are so popular in face-to*face teaching. 



Paradif^m: 


'Instruction 


'Conversation 


OBJECTIVES: (OUTPUT) 
(why?) 


specific 
pr&Kkfined 
products 
standard 


general 
negotiable 
processes 
variable 


MESSAGES: GNPUT) 

(what?) 

(when?) 

(who?) 

(whom?) 


designed 
pre-prcpared 
instructor 
one-to-many 


creaicd 
on-line 
participants 
many-to-many 


INTERACTION: (PROCESS) 

(focus) 

(analysis) 

(feedback) 

(complexity) 


behaviors 
crilcrion-ref 
coiTCCtive 
one-layer thick 


ideas 

content^ structure 
constructive 
interwoven layers 


DISTANCE EDUCATION: 
example 


Correspondence courses 
Ccinputer Mediated 
Communication (CMC) 


Teleconferencing 
Videoconferencing 



Table 1. Instruction verses conversation 
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Finally, as Table 1 illustrates, teleconferencing fits 
naturally on the right hand side of the 
instructional/conversational continuum and so may be 
used in subject areas where many would consider the 
instructional paradigm to be less than ideal. 

Computer-Mediated Communication 

Sending messages from one terminal to another, even 
over international boundaries, is not new. Boyd (1990) 
points out that electronic mail and file transfer is as old 
as mainframe computers and was already quite 
widespread among scientists in the 1960s. However, 
only recently has it become a popular and relatively 
widespread means of communication for educational 
purposes. Judging by the quantity of literature on the 
topic that has been published in the last few years, this 
may well be the fastest growing area of application of 
technology to communication and education. Perhaps 
the most common way of using computer -mediated 
communication is for so-called computer conferences. 
These have many of the characteristics of 
teleconferences, except for the real-time aspect 

The asynchronous nature of the communication process 
in CMC, has been found of educational value in 
interactions between persons who are prevented from 
meeting face to face by distance. At Syracuse University 
we have, for some time, been supplementing classroom 
discussions by extended electronic mail interactions 
between students who are resident on campus and others 
who live further away. We have found that the e-mail 
discussions are often much deeper, much richer and 
involve a larger proportion of the student group than is 
typical in a conventional class-based discussion 
(Grabowski et. al., 1990). However, we have also found 
that the use of e-mail as such a discussion medium 
poses certain challenges to the instructor. Notable 
among these are keeping control over the direction of 
the discussion and helping participants to maintain an 
overall view of the structure of the debate, as it evolves 
over time. Several instructional strategics for dealing 
with these challenges have been experimented 
(Romiszowski & Jost, 1989). 

There are many benefits claimed for using computer 
mediated communication in education: a democratic 
environment for group interaction and learning; a record 
of the entire course; students don't miss any of the 
instruction due to absenteeism or daydreaming; more 
students respond to questions and ideas; the responses 
are longer and more complex than in a traditional 
classroom; Uie students learn from each other; teacher 
comments become more interactive (Romiszowski & 
Jost, 1989). 

A problem Uiat needs to be addressed when using CMC 
as an integral part of a course, is that students may, at 
any lime, be inpuiiing comments related to different 
stages of the development of the topic. Not only is the 
discourse "multi-level" in that several different topics 
may be in simultaneous discussion, but it is also 



"multi-speed", in tiiat participants are responding to 
various phases in die evolving argument, at any one 
time. A lot of reading and rereading is required to keep a 
clear structure of the discoiu^ in mind. (Romiszowski 
& de Haas, 1989). 

Another problem originates from the distance 
communication aspect of CMC. Although distance 
communication allows one to participate in a discourse 
which may otherwise be impossible, it also introduces 
some difficulties of control of the discourse. The 
instructor loses some of the benefits offered by a face- 
to-face group situation. When the discussion drifts off 
the topic, it often takes longer, and is more difficult, to 
bring the group back on task (Romiszowski and Jost, 
1989). 

Initial Experiences Using CMC 
for Instruction 

Our own experience in the use of computer-mediated 
communication as a formal methodology of instruction 
commenced with a short computer-conference (or rather 
SEMINAR) which linked up two student groups on 
similar courses al the Universities of Twenie (in 
Holland) and Syracuse (in the USA). Our interest was in 
using a simple system (the existing e-mail linkup 
Uirough Bitnet) but with a greater amount of structure 
to the interactions, that would replicate ceitain face-to- 
face strategies of instruction. It was decided to replicate 
the SEMINAR structure, as this would seem to be 
particularly suitable to the characteristics of the system 
(Romiszowski & de Haas, 1989). Rather surprisingly, 
the discussion that did take place was quite different 
from the intentions defined in the initial seminar paper. 
A retrospective study of the dynamics of the seminar 
suggests that the initial respondents to the paper, who 
chose to challenge the basic assumptions, then set the 
tone for the whole in that later respondents tended to 
reply to the latest messages on the system, rather than 
to a general framework for the discussion that was set 
up in a paper prior to the conference. 

Separate analyses of the structure of the discussion were 
performed, retrospectively, at both Twente and 
Syracuse, by different participants. One aspect of 
interest was that the structure was seen to be different 
by each of the analysts. This finding illustrated the 
somewhat personal view that each participant developed 
of the seminar. 

The observations described above were the basis for 
planning a second somewhat larger CMC seminar, held 
during die months of April and May 1989. This was 
planned to more closely examine some of the factors 
that had been observed earlier (Romiszowski & de Haas, 
1989). Once more the discussion rapidly drifted a-vay 
from the expected theme initially defined for the seminar 
and a coherent view of the seminar's structure was lost. 

In May and June of 1989, a third experiment was 
conducted, this time imposing more precise, course- 
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related structure to the proceedings. The ensuing debate 
was a much richer and deeper, as well as a much more 
focused experience. In addition, the depth and detail of 
this ensuing discussion compares very favorably with 
that usually obtained in a classroom based seminar 
session. 

We observed, however, that the style of contributions 
and their length had undergone a significant change. 
Whereas the messages on the previous occasions were 
generally short and chatty and directed to one or other of 
the recently preceding e-mail messages, the responses 
now became longer, more formal and precisely tied to 
specific points raised in the original paper. 

These differences have both their positive and negative 
aspects. On the positive side, the intensity of 
involvement and participation increased. Also, the 
discussion did not wander disastrously away from the 
topic initially set. In some eyes, of course, this 
focusing could be considered a negative aspect, in that it 
may have limited the exploration of **bright ideas** that 
were not foreseen in the original author's focus 
questions. But from an instructional design viewpoint, 
some focusing (more than was achieved in the previous 
two experiments) is generally desirable. 

Throughout this series of experiments, much attention 
was paid to the question of how a participant may form 
and maintain an overall structure of the discourse that is 
evolving over time. Several methods were used for 
keeping track of structure. One limitation of this 
system is that one cannot construct a hierarchical (or 
any other) structure of notebooks. Any complicated 
structure must be noted separately, off line. 

It is in this context that a Hypertext environment, or 
shell, into which all messages could be stored in the 
most fully cross-referencable manner, suggests itself as 
a possible solution. 

Hypertext and its Contribution to CMC 

The collaborative annotation of student papers by both 
faculty and students is something that many proponents 
of hypertext advocate and (occasionally) practice. One of 
the more long-lived working hypertext environments — 
Intermedia at Brown University — was specifically 
designed to support this son of collaborative educational 
activity (Yankelovich et al, 1988; Landow, 1987). 

However, current hypcnext systems which can support 
this type of activity effectively are large, costly, and not 
easily compatible across different computer systems. 
The Intermedia system is "built on top of the 4.2 BSD 
UNIX operating system and runs on IBM RT/PC and 
Sun workstations which support Sun's Network File 
System (NFS)" using a special version of the C 
programming language, and a host of specially 
developed applications packages (Meyrowitz, 1986). 
Other powerful and flexible systems, such as 
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NEPTUNE (Delisle et al 1986) are quite different in 
configuration, but no less complex. 

Our current projects are based on the concept of 
utilizing, as much as is possible, what already exists 
and is established in significant quantity in the 
educational marketplace. At this point in time (in USA 
universities at any rate) the MS-DOS personal computer 
has the largest market share followed by the Macintosh. 
Also, many existing e-mail systems operate as 
networks of existing general purpose personal 
computers. When used for e-mail, these computers are 
called on to act simply as dumb terminals linked to the 
mainframe that supports the communications system. 
However, it is possible to make use of the presence of 
local power and memory storage to download incoming 
mail messages to be stored directly on disc. 

In our CMC conference's and seminars using Email, no 
attempt was made to impose a centralized suucture 
through the system. Indeed, the limitations of the Email 
systems we have prevented us from doing this. The 
instructors involved would attempt to define an initial 
structure in an opening "position paper". We then 
encouraged each of the participants to map their own 
view of the emerging structure. Students organized 
course materials into notebooks that were personally 
meaningful. Students can also be provided with some 
basic initial structure which separates course content 
from personal communications. They can also be given 
some suggestions for organization. One time-saving aid 
to understanding and organization is to encourage 
students to title their responses so that others are able to 
quickly recognize what the note's content is in response 
to. 

Further organizational tools which help the student in 
following the often interweaving subtopics, which 
emerge, is to download and organize messages and to 
create pattern notes of the topic areas which show the 
interrelationships that may exist. The students can also 
benefit by comparing their suncturcs with each other. 
Instructors can organize related topics, ideas, etc. and 
send them to students for additional discussion. The 
students active involvement with the content structure 
leads to a more meaningful understanding and greater 
transfer of the new knowledge. 

It should be possible to adapt an existing pc-based 
hypertext environment so that new messages can not 
only be stored in approprb.te subdivisions of the basic 
su-ucture one creates, but can also be freely cross 
referenced to other relevant messages. We are 
investigating the applicability of some currently 
available and relatively cheap hypertext applications 
packages for this purpose. 

Experiment #1: 

Seminars In a HyperCard Environment 

A recent experiment involved the use of a networked 
communication space, set up in HyperCard and running 
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on a Macintosh cluster. Each of the participants in the 
CMC seminar has a slack of pre-preparcd cards, with 
scrolling text windows, thus giving unlimited writing 
space, but dividing the text into "chunks", so that each 
separate issue/topic in the discussion has its own card. 
These cards can be accessed by all the participants, who 
can read them, add small annotations and make links to 
cards in their own stack. As the discussion progresses, a 
cross- referenced network of documents is created. This 
facilitates the review of the discussion by a new 
participant or by someone who has not logged on for 
some time. It also ensures that new comments are 
linked to earlier ones on the same topic, as well as 
making it very easy to review all the contributions of 
anyone participant. 

Figure 1 illustrates the home card of the system set up 
for seven seminar participants. In addition to the seven 
personal stacks, users have available a short resume of 
each participant and their phone, fax and e-mail 
numbers, in case they want to get into touch 
personally, send a reprint of an article, etc. Clicking on 
the required icon takes one directly to the first card of 
that stack. Browsing through that stack, the reader can 
identify the specific topic under discussion by a label in 
the top left comer. Figure 2 shows a typical card from 
one of the slacks. The slack "owner's" comments (in 
this case on an experiment performed with an expert 
system) have in turn been commented by another 
participant, who has registered the card number in his 
stack where the relevant comment can be found, in the 
small window below his personal icon. To read the 
comment, just click on the person's icon and key in the 
card number. To register a new comment, just type it 
into one of your own cards and create a cross reference 
by typing that card's number in your window on the 
other person's card. Figure 3 shows the comment made 
by another student on the question in Figure 2. The 
system was designed by one of the authors, Echeol 
Change specifically so that no learning of HyperCard or 
any complex access commands would be necessary. 
People arc comfortable on the system in a matter of a 
few minutes. 

We are currently using this networked system on 
campus, in an experiment which is a model for a 
system that could in future be accessed over 
telecommunication links from any distance. As the 
system is conceived as an alternative to e-mail or other 
readily available CMC systems, we are concentrating on 
the comparison of these alternatives from the user's 
viewpoint. We have been using several alternative 
systems: the system just described and a file-sharing 
system in which all participants comment a position 
paper in the same file, but do so by inserting their 
comments among other related comments. This can be 
implemented on existing mainframe systems, in our 
case in CMS. Users have access to a read-only initial 
file (the paper to be commented) and to a read/write 
comments file. These can be set up in two scrolling 
windows, on a split-screen, so that comments can be 
typed in as the paper is being read. Organization is 
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achieved simply by referring comments to line numbers 
in the original position paper. This approach has the 
advantage that it is already widely available, networked 
internationally through Bitnet or its alternatives. It has 
the disadvantage that it is clumsy to use and much 
learning is necessary to become skilled in its use. 
Surprisingly, however, users do not have much of a 
problem with the interpretation or generation of 
comments that are mixed in with other users* comments 
in one document We have therefore prepared a user 
friendly, Macintosh equivalent of this system which we 
are currently testing in comparison to the networked 
system described earlier and to the mainframe-based 
system. Figure 4 shows a typical screen from this 
system. 

All these alternative systems have been used by students 
to present and discuss their seminar papers. During one 
semester, each student had to perform a bibliographic 
research study and a practical evaluation on a selected 
aspect of new technologies. The results of this work, 
together with a series of key issues for discussion, were 
then presented as a seminar paper to the rest of the 
group. The discussion of the seminar papers took place 
on one or other of the CMC systems. In all, seven 
seminars were held, six generated by the students and a 
final course review seminar set up by the present 
authors. The characteristics of these seminars were as 
follows: 

1. Computer Mediated Communication**, 
addressing 8 key issues. Woiked as a pair of 
files on the mainframe (position paper and 
communal comments space). Set up in CMS 
and accessible from any campus computer 
cluster or from home via modem. 

2. '*Hypertext for Learning**, addressing 6 key 
issues. Worked as as a pair of split-screen 
scrolling windows in a HyperCard slack (see 
Figure 4). From the user*s viewpoint this 
configuration was identical to the CMS 
mainframe-based system, with the added 
characteristic that each key issue of the 
position paper, and therefore all the related 
comments as well, occupy a separate card in 
the stack. 

3. **Expcrt Systems as Job Aids**, addressing 5 
key issues. 

4. ''Optical Videodisc Technology**, addressing 5 
key issues. 

5. **Computer based Simulations**, addressing 6 
key issues. The last three ran on the hypertext- 
like network, in which each participant has 
his/her own workspace in a separate slack (see 
Figrres 1 to 3), but can simply link any card 
to any other card in any of the other slacks. 

6. "Knowledge Acquisition and Representation", 
addressing 6 issues. Worked as a CMS file on 
the mainframe, but without scrolling windows. 
Users simply open space and insert their 
comments into the original paper at the 
appropriate place. Each comment is identified 
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About this class activity: 

This class activity was designed of using two versions , 
machine-based and paper-based of a small expert system 
call CBT Analyst to get advice of selectin9 courses tor CBT. 
Four course contents ( scenarios ) were given to six 
participants who were devided into two groups, group A 
and group B. Few statements of course contents were 
introduced at the beginning of each scenario. Base on the 
same course content, participants were asked to consult 
from machine-based or oar^r-based expert system. 
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Figure 2. The fifth card in a seminar on expeit systems. 
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Topic : Cognitive Psy /Science 
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THE ENCYCLOPEDIA NEEDS A BETTER INDEX, BUT 
CONSIDERING THE CONTENTS, .WHICH ARE MOSTLY 
HISTORICAL EVENTS, MAYBE A CHRONOLOGICAL ORDER DOES 
MAKE SENSE. IT DIDN'T HELP US BECAUSE WE WERE NOT 
USING THE SET FOR ITS INTENDED PURPOSES; WE WERE NOT 
LOOKING FOR HISTORICAL EVENTS AND PEOPLE WHO ARE 
POPULAR, BUT FOR THINGS WE KNEW ABOUT, WHICH MOST 
USERS WOULD NOT BE LOOKING FOR. 
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Figure 3. A comment card on the Question poaed in Rguie 2. 
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Ted Nebon, vho coined the void in the 1960s, defines "hyper" as 
"dxtended, genenli2ed, and multidijfnensional" (1973). Michael Helm vxltes, 
"text derives oxielnally from the Latin void for veavint and for intervoven 
iKAterial, and it comes to have anextraordinaiy acciuacy of meanine in Ha 
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I think hypertext reading 15 much like skipping around in a book, reading; some here and some 
there. The advantage of hypertext over a book in doin^ this is 2 things: t tendency of people 
Krtth books not to read in this manner, even though they could; and a protilem both hypertext 
and books have*if no good vay exist? to find things, effective and efficient brovsing can't 
happen (i.e., indexes, etc.}* 

^ 

Figure 4* The 9lit-scieen» scsoUing window version. 
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The choice and sequence of CMC systems used were 
largely dictated by (M^ctical and timeline considerations. 
The last two seminars returned to using mainframe- 
based systems, as accessibility from anywhere, both on 
and off campus, proved to be one of the most important 
student requirements. The alternative system 
configurations were employed to overcome some of the 
**uscr unfriendliness** of the initially used split-screen 
arrangement. Students were unwilling to go to the 
trouble of setting up two CMS files as two scrolling 
windows, on screen at the same time, a procedure that 
required a sequence of some half dozen commands. 
Instead, they invariably printed off the position paper 
and used the hard copy alongside the computer terminal 
for reference as they worked on the comments file using 
the full screen display. The last two seminars attempted 
to facilitate the use of a mainframe-based system, 
adapting to preferred student working habits (and in the 
process sacrificing some of the structural features that 
were originally designed into the experimental CMC 
systems). 

The five alternative systems were compared by 
observing the student::* working habits and interviewing 
the students on their preferences. Also, some attempt 
was made to measure the amount and the quality of 
interactive/collaborative work on the seminars. 

In relation to working habits, it was observed that the 
majority of the students did not actually choose to work 
on-line. Five of the six course members would generate 
hard copy printouts of the latest version of the seminar 
discussion, would study these ofT-Une and compose their 
comments. Only then would they log on again in order 
to input the comments. They would do this not only 
with tiie mainframe-based files, but also with the 
Macintosh-based hypertext-like databases. Part of the 
reason for this may have been that the Macintoshes had 
to be accessed by going to the School of Education (and 
often encountering a line of other students waiting to 
use them). But the students also expressed a preference 
for working ofT-Iine, rather than spending a co.isiderable 
amount of time peering at a computer screen. In 
addition, they wished to keep a hard copy of their work, 
as they did not feel much **ownership** of the communal 
files in which they were working on these seminars. 
This was one unexpected aspect of our change from 
using an e-mail system where each student has a 
separate account, to a shared account open to all the 
course participants. 

In relation to students* observations, perceptions and 
preferences, it is worth mentioning the following: 
The majority of the group did not like using the 
hypertext like environment pn3vided for three of the 
seminars, mainly because of the extra work 
involved in clicking on icons in order to move 
from one card to another. No one had any difficulty 
in learning or remembering how to navigate around 
the network, but they resented the need to spend 
time doing so. 



Also they did not feel that the networked system 
helped them to form a ckarer picture of the 
structure of the seminar discussion. Whilst some 
felt that the act of creating links between cards 
helped them to better visualize the structure of the 
topic under analysis, most considered that the 
browsing of a network of cards left them without as 
clear an idea of structure as seeing all the related 
comments chunked together in one file. They also 
felt that there was a time penalty in using the 
networked system, not so much because of the 
logistics of moving from card to caru, but because 
they often felt the need to return to a card just to 
reread it, as it was out of sight as they were trying 
to compose a reaction to it. Research by Monk et. 
al. (1988) supports the view that there may well be 
a significant time penalty in using hypertext 
documents as compared to scrolling files. 
Among the special benefits of the approach, some 
students mentioned that the discipline of having to 
write down one*s comments not only records 
creative observations which would otherwise soon 
be forgotten, but actually encourages the generation 
of creative ir'^ights into the topic. A second 
benefit, directly related to the asynchronous nature 
of the process, was quoted by three of the four 
overseas students in the group • being able to take 
one's time off-line, to study ,ers' and compose 
one's own contributions, goes a long way to 
breaking down the barrier to participation that is 
created by incomplete mastery or lack of confidence 
in using English as a second language. The figures 
presented in Table 2 bear this out • though the 
overseas students may have participated somewhat 
less intensively than the USA students, they have 
nevertheless all participated, something that seldom 
occurs in face to face seminar discussions. 
In studying the figures shown in Table 2, it may be 
helpful to note that the seminars were scheduled at 
one-week intervals, except for the first one. which 
ran for a full two weeks before the second one was 
launched. This may account for the higher level of 
interaction (comments per student) than the general 
trend would suggest. Also, the last seminar, being 
instigated by the professor and having an overall 
review and synthesis function across several topics 
previously discussed in earlier seminars, probably 
generated a higher than average propensity to 
participate and to respond profusely. 
The figures in Table 2, although based on a 
relatively small group, do illustrate a very healthy 
level of participation, particularly in the later 
seminars. Even in the earlier seminars, the total 
amount of discussion compared favorably with the 
amount of interaction generally obtained in a one or 
two hour face-lo-face seminar. As the semester 
progressed, the level and intensity of participation 
continued to increase, as shown by the growing 
figures for ''comments made per person per issue 
addressed*' and "length of comments made per 
person per issue**. This latter figure steadily 
increases till it stands some eight times as high at 



425 



Romiszowski, Josu and Chang 

the end t>f the semester as compared to the 
beginning. The average comment in the first 
seminar was two lines long. Mostly, these 
comments were cryptic agreements/disagreements 
or questions for further explanation. These in turn 
called for a lot of time and effort to moderate the 
seminar and respond to the questions being asked. 
Note that in the first seminar.the volume of 
moderators' comments (paper author*s and course 
professor's) were more than double the volume of 
all the students comments together. This number 
falls progressively, until finally we are able to 
eliminate the moderation function altogether. 
• What the table 2 figures do not show is why it was 
possible to reduce and finally eliminate the 
moderator's role. The quality of the contributions 
made by the participants progressively improved as 
they became more familiar with the use of CMC. 
They became more incisive, more argumentative, 
more confident in taking issue with another 
student's contribution, etc. The group became quite 
able to lead and to moderate its own discussion. 
Thus the increasing volume of participation during 
the course of successive seminars, represents a 
learning process on the part of the participants, 
developing their own style of print-based 
discussion. !: was particularly gratifying to observe 



the growth in capacity to offer and lo receive 
constructive criticism, especially on the part of 
certain overseas students in the group who, in 
addition to the language inhibitions mentioned 
earlier, often have strong cultural inhibitions that 
effectively prevent them from engaging 
successfully in the give and take of critical analysis 
of each other's productive thinking. 

Experiment «2: The Use of Case Studies 
In a HyperCard Environment 

Over the last few years, we have been developing a 
range of case materials that can be used for the teaching 
of instructional design and development skills. In 
developing these case studies, we are, of course, 
attempting to pracdce the technology of case writing 
and case discussion as it is currently understood We are 
also interested in implementing innovations, 
particularly along the lines of the creation of automated 
case study exercises that could be used in distance 
education as well as in classroom based instructional 
situations. 

The instructional design and development case 
collection covers a wide range of skills that needs to be 
developed in a practicing instructional designer. 



Seminar #/ 
Title 


1 

(CMC) 


2 

Hypertext 


3 

Expert 
System 


4 

Videodisc 


5 

Simulation 


6 

Knowledge 
EnKineerinK 


7 

Review 


# Issues 
Addressed 


8 


6 


5 


5 


6 


6 


9 


U Comments 
Made 


59 


24 


17 


23 


17 


36 


79 


# Lines of 
Comment 


128 


104 


96 


219 


134 


310 


1412 


Avg. Comment 
LenRth 


2 


4.5 


5.5 


10 


8 


8.5 


17.5 


# Comments/ 
Student 


11.8 


4.8 


3.4 


4.6 


5.7 


7.2 


13.2 


# Comments/ 
Std/Issue 


1.5 


0.8 


0.7 


0.9 


0.9 


1.2 


1.5 


(US Studeriu?) 


(1.75) 


(0.75) 


(1.0) 


(0.9) 


(0.8) 


(2.5) 


(2.2) 


(Overseas) 


(1.4) 


(0.83) 


(0.5) 


(0.9) 


(1.0) 


(0.9) 


(1.1) 


#Lincs/ 
Student 


25.6 


20.8 


19.2 


43.8 


44.7 


62 


235 


# Lines/Std/ 
Issue 


3.2 


3.5 


3.8 


8:8 


7.4 


10.3 


26.2 


(US Students) 


(3.0) 


(4.6) 


(6.3) 


(12.4) 


(5 8) 


(16.0) 


(35.9) 


(Overseas) 


(3.25) 


(2.7) 


(2.2) 


(6.3) 


(8.25) 


(8.9) 


(21.3) 


U Moderator 
Mcssaces 


63 


22 


5 


8 


5 


3 


0 


# Moderator 
Lines 




143 


65 


125 


63 


30 


0 


Mod/Std 
Comments (%) 


233% 


138% 


68% 


57% 


47% 


10% 


0 



Table 2. Data From Initial Experiment with The Hypertext Discussion Environment. 
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One particular skill area of great importance is what we 
term front-end analysis. This is the analysis that ought 
to be performed in order to ensure that a particular 
training project really starts off on the correct lines. 
Several cases have tx^n developed to help our students 
learn these skills in a practical context. One of these 
cases will be used here as an example of the general 
approach we arc using. 

The readers are invited lo imagine that they have been 
called to bid on a project by a local university that has 
perceived a problem in the area of the skills of its 
faculty memhNcrs. You arrive to an initial meeting where 
you are briefed on the background to this supposed 
problem. You learn thai the university has invested 
quite heavily over the last few years in a variety of 
audiovisual aids, among them a large number of 
overhead projectors, one of which has been installed 
pcnnanently in every teaching space of the campus. A 
recent survey has shown that since this has occurred, 
very little increase in the use of the overhead projector 
has in fact taken place. Very few faculty members ever 
use the overhead projector at all, and incidentally many 
of those who do use it use it ineffectively or badly. The 
screen cannot be seen from certain parts of the room; 
the incident lighting is badly organized; the 
transparencies are so prepared that they are illegible, etc. 
You arc being invited to 'teach our faculty to use the 
overhead projector and to use it weir. 

In the pastt this situation would be developed as a role 
play exercise in the classroom* a number of typical 
**stakeholders** in the problem and interested parties 
being represented by the professor and assistants. The 
students on the course could interview several faculty 
members, the audiovisual department employees and 
gather whatever data they felt was necessary in order to 
bc;tcr understand the problem, its underlying causes and 
therefore evaluate to what extent the opining that's 
being requested could solve the real problem. All of this 
infonnation would then be used as a basis for the 
preparation of a recommendation to the client on what 
really should le done about the problem. The process of 
generating this recommendation would, in the 
classroom implementation, be carried out by small 
group discussions and seminar-like reports to the rest of 
the group. Here we have a typical and fairiy classic use 
of the case methodology in the classroom situation. 

In preparing a case like this for publication, we face two 
problems. First, the data gathering stage of interviewing 
faculty and other interested parties is a very dynamic 
student led exercise when performed in the role play 
context as described above. When this is uansformed 
into wriuen case materials, we have the uanscripts of 
discussion$/interviews with a variety of people, which 
have to be read at length by the student and in this 
process all the studcnt-directcdness of the activity is 
lost. The skills of case writing come into play in order 
to write scenarios that donH give the game away too 
much and yet which do include all the information that 
students are expected to utilize as a basis for their 



recommendations to management. The second problem 
is connected with the case discussion. Most published 
case materials do not actually insoiiment the discussion 
but resuict themselves to the case description, the data, 
and the seuing of some problems for discussion. The 
discussion organization or implementation is up to the 
leader of the case discussion in the classroom situation. 
If the case is being studied individually by a reader of 
the text, then it*s really up to the reader to organize it 
for him or herself. Our interest in some of the case 
studies we have been developing is to take the 
insurumentation through to the case discussion stage.. 

To do this, we turned to a methodology developed first 
in the United Kingdom in the late 60's and early 70 s 
and called Suuctural Communication. This was an 
attempt to create a discussion environment for open- 
ended self study of topics where multiple viewpomts " 
and various possible correct responses were the rule. 
The methodology was applied in many different 
contexts during the early 70's including an experimental 
correspondence course in the Harvard Business Review, 
where a Harvard case was discussed by readers 
responding to a pre-structured programmed exercise 
(Hodgson and Dill, 1971). Most of these eariy 
implementations were in the form of paper texi-boscd 
self-study exercises, but some used mainframe 
computers as a delivery medium. With current 
technology, the Structural Communication 
methodology has become a powerful and very practical 
approach to the authoring of conversational CAI 
tutorials ( Romiszowski, Grabowski, and Pusch, 1988). 

The components of a structural communication study 
unit as first formulated by Benneu and Hodgson 
(Hodgson, 1974) are the intention, presentation, 
investigation, response matrix, discussion guide, 
discussion comments, and finally viewpoints (see 
Figure 9). Each of these plays a part in engaging the 
reader in a deep analysis nf the problem that has been 
presented. In our case, the intention and presentation 
were taken care of by the case description. 

The response matrix was an array of 20 well known 
approaches to the solution of job-performance problems 
(See Figure 10). The decision-making model used in 
this exercise is based on the heuristics for front-end 
analysis suggested by Romiszowski (198 1). The student 
can select any combination of items. The number of 
different possible responses that a student can make are 
very large, in fact, running into the millions. If one 
restricts oneself to plausible responses, the number may 
be a lot smaller but still very large. Analysis of the 
various response pauems is prepared in advance by the 
authors of the exercise on the basis of significant 
inclusions or exclusions of items, ote that this analysis 
is the **script** from which a case study decision leader 
could speak if discussing the particular discussion "live** 
or altemaavely from which an author could work in 
order to write some pre-prepared comments to further 
elucidate and discuss particular aspects o^ a given 
student's response to a given problem. This latter 
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approach was- used in our work, producing a set of 
several dozen pre-prcpared comments lhal would be 
directed in different combinations to different students, 
depending on how they responded. The style of these 
pre*preparcd comments is open ended and encourages 
reflcclive thought rather than the correction of wrong 
responses. We may say that we are attempting to 
provide constructive comment rather than corrective 
feedback. 

The implementation of such case studies in an 
electronic format is desirable for at least two reasons. 
Firstly, this will overcome some of the utilization 
difficulties that extensive use of such an interactive 
n^ethodology has when implemented in a paper format 
Secondly, electronic implementation of such exercises 
should enable one to open them up for student generated 
comment and input in a way that a published and 
printed version can never do. By the combination of 
electronic computer mediated communication with the 
electronic storage of the basic case-study materials, it 
should be possible to create an exercise which, in 'iie 
initial stages, presents authors* prepared views and 
comments and then allows students to comment those 
comments and actually converse with the author, 
although that author may be located somewhere at a 
distance. Current research that we are engaged in at 
Syracuse University is exploring the viability of this 
approach. We have developed some computer software 
that can implement the Stn^ctural Communication 
methodology, in HyperCard on the Macintosh computer 
and also more simply on mainframe systems that can be 
networked worldwide if necessary. 

We shall use the Macintosh system here as an 
illustration because of its better visual qualities. The 
case study mentioned above has been implemented in 
HyperCard, the data gathering stage and the problem 
solution stage being handled by two separate pieces of 
interlinked software. For data gathering, we have 
employed the previously developed Hypertext 
environment, where information about the problem that 
was posed in the case is stored as a network under 
student control. By doing this, we are avoiding the 
degradation of the role-play exercise when presented as 
printed transcript, because the student is still required to 
decide what type of information he or she thinks is 
pertinent and follow this through as a trail from one 
card to another in the "maze" of information that has 
been created about the background to the case. Students 
can, if they insist, read all the information in the data 
base in a more or less linear sequential way. However, 
this would be very time consuming. Most students 
choose to follow particular trails through the network of 
information, and in order to do this usefully, are forced 
to formulate in their minds the question that they are 
trying to follow up before they actually start doing any 
reading. 

Figure 5 illustrates the problem that has bc^n set and 
shows how a student can go either to the solution or to 
the case study sections at will. Figure 6 shows the front 



card of the case study section which introduces the 
student to 7 key figures who can be interviewed and also 
to the resumes of these 7 people (stored in the "who's 
who** cards indexed along the right hand side of Uhe 
screen). By clicking the mouse on any of the "who's 
who" cards, the student can open the resume of that 
particular person and get a general idea of what their 
function is in the university, etc. By clicking on the 
name of that person on the star shaped diagram in the 
middle of the screen, the student opens a conversation 
with that person which is presented over a number of 
cards in a stack. Each of the cards, however, deals with a 
particular issue, some of these issues being relevant and 
others irrelevant to thie solution of the problem that was 
originally posed. Figure 7 shows a typical card, in fact 
one from the interview with the client 

The scrolling window in the upper part of this card 
contains the transcript of a discussion between the 
analyst and the particular person being interviewed. At 
the bottom of the card, there are a series of icons that 
enable this card to be linked to relevant cards in the 
interview stacks of all the other six available 
participants in this automated role play. We note that 
there is relevant information in card 3 of the stack 
named "Bean" and also in card 7 of the stack named 
"Egan." The student immediately knows that relevant 
trails of inquiry can be commenced by clicking onto 
titose stacks. If the student considers the card currently 
being read as supplying information that is important 
for the solution of the problem, then this would suggest 
following trails to other relevant information at that 
point in order to immediately formulate a position in 
regards to that aspect of the problem. If on the other 
hand the student feels that the issue being dealt with on 
the current card is not very relevant to solving the 
problem, then a lot of reading can be avoided by not 
clicking on the subsidiary cards and indeed continuing 
directly to another card on a different aspect of the 
problem altogether. This inquiry environment created by 
a network of information cards is an attempt to give 
back to the student the control over what questions to 
ask and what information to follow up. 

When *he student has gathered as much information as 
he or she thinks is relevant and has formulated a clear 
idea of what are the underlying causes of the problem 
and what solutions may be most appropriate, a click on 
the button labeled "solution" takes the student to 
another part of the software which is prefaced by a card 
entitled "Structural Communication" (See Figure 8). 
The student can at this point click on other buttons 
which give some background to structural 
communication methodology and how to use it as a 
learning device. One of these cards is shown in Figure 
9. The main button, however, that a student should 
click on is the one labeled "exercise," for a presentation 
of the response matrix. The response matrix for this 
particular case is shown in Figure 10. This presents a 
total of 20 types of solutions that could be applied in 
some combination to the problem that the case has 
posed. The student can select any combination of these 
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20 by clidQng on the appropriate buttons. The choice 
made appears at the bottom of the card 

Once the stuilem has reqx)oded, the program analyzes 
this req)onse and presents an index to the conmients 
a vailabk phis a suggestion of ii^iich of those coomients 
may be appropriate for that particular student to read. 
This is presented on a card illustrated in Figure 11 
which acts as the diKussion guide for the remainder of 
the exercise. The student can now dick on any of the 
icons and go to the related comments. Note that the 
student is not restricted to clicking on those comments 
which are diagnosed as being ^)propriaie but can )o6k 
up any of the comments that are available. However, 
those that the author conskkrs as appropriate 10 the way 



H^Mrtert Seminars 

the student handled the problem wodd be the ones that 
normally tL student would read first On clicking any 
one of these, the student will automatically be taten lo 
a comments card like the one illustrated in Figure 12. 
Here there is a scrolling window in which the author's 
comment is presenied. Figure 12 shows a comment 
genmted by the student NOT including item Q Grom 
the reqxmse matrix ^gure IQ). Note that diere is no 
physical restriction on the length of this comment It 
can be very much moro than one screen of lext if tluit is 
required. Note also that the style of the comment is 
open ended. It takes the student beyond the excfdae in 
order to reflect more deqdy about the implicatioiis of 
Ae partKuIar positkm that hu been adopted* 



Till- PRO 




You have been invited to make a proposal to 
I the XYZ University. The head of faculty 
development, FRED ASPEN, has called you in as 
a consultant to " TRAIN THE FACULTY MEMBERS 
TO USE THE OVERHEAD PROJECTOR IN THEIR 
CLASSES, AND TO USE IT WELL". Apparently, a 
recent survey showed that few of the faculty 
I make use of the O.H.P.'s, and those who do, often 
e it badly. You, as a performance technologist. 



CASI 
SlUDY 



')0I II 1 ION 



Figure 5. Initial card of the OMP case study. 
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EXIT ^^^^^'^'r'oi wnpsmjiTWN 

Figure 6. The HypaCard-feisedcase study materials. 



Topic ; 



Fred Aspen 

CONSULTANT: Hello Dr. Aspen. Before we go any further, 
could you give nne some more background to the problem 
before us ~ as you see it. Why exactly do you see the need 
to train faculty in the use of the overhead projector? 



ASPEN: O.K. Let me give you some background. About 2 
years ago, the audiovisual services unit (which reports to 
me) requested the purchase of more overhead projectors. 
The system at that time was centralized and faculty had to 
reserve an OHP in advance from one of the three service 
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Personal Comment 



Bean Ford Davis 



Egan Goody Clark 




Figure 7. Typical card from an interview script. 
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STRUCTURAL 



COMMUNICATION 



''\^iS'^?-^-?-.' r^EY FfiATUI^S; DIfSCRIPTIONi .ST^T EXERCISE^ 



ma 



FiftireS»Hdro&dlirdoCdie€«se-di^^ ; >v 



* Key Features i' 



..^ u;;^ f initial study/simulation/real-life material 



' * probleol s*ets * 



response matrix " 



discussion guide . 



* discussion comments 



Fifure 9* The cdmpooents of a Sinictuial Commumcatton exercise* 



ERIC 



431 



Romiszowski, lost, and Chang 

I Select all actions you votntiir^econiimeml. (Click the letter in the hox.) \ 



m 


Re-structure the job 


m 


Enrich the job 


11 


Re-as$ign/tran3fer 


m 


Plan feedback of results* good perf. 




Teach prerequisites 


B 


increase the rewards for good perf. 


m 


Formel instruction 


E 


Eliminate the rcvards for good perf. 


m 




m 


Eliminate the punishing conseg. 


B 


Telling/shoving 


m 


Increase punishment for nen-perf. 


m 


ri«n prwiice. 
Re-trelii frefiu. 


m 


Re-plan the equipment or voricspace 


m 


Supply a manual 


m 


Re-plan the voric methods/sequence 


m 


Supply Job aids 


m 


Re-erg. the respenslMlitles 




Enlarge the job. 


m 


Re-org. the sgstems or management 



QUIT 



Your choices... 



NEW 



iA B D 



H 



M 



If doncX 

cucic ny 



El 



[here are study comments for YOU!) 




Click it Click It Click It Click It 



Click It 




Click it 



Click It Click it Click It 



□ 



Figure 11. The Discussion Guide! 
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conriENTs 



You did not think it necessary to replan the working 
equipment or workspace. If you mean the projector, we 
agree. However bad the design of the projectors that the 
institution has bought, it would be impractical to suggest 
they scrap them all and replace with a more ergonomically 
user-friendly model. If you mean the classroom space/ 
layout, we do not agree. Some very simple actions can 



pi 

r 
I 
If 

ill 

1 
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Figure 12* One of the 13 Discussion Comments 

Note particularly, however, that the card presents a 
second scrolling window, entitled **Your Comment,** in 
which the student can type in any message to the author 
requesting further information, registering disagreement 
with the author*s views or whatever comes to mind 
This facility allows a discussion to go a stage beyond 
the original structural communication type of exercise. 
In the Macintosh implemented version, these comments 
are stored in their re4)ective cards and can be browsed by 
the author or indeed by other participating students. 
Very often a student may make a comment which is 
prompted not only by the authc^s viewpoints but also 
by viewpoints that have been put in by a {M-evious 
student. In this way, a discussion gets going. The 
discussion develops at a deep level, beyond the initial 
analysis of the case material. 

In the mainframe implementation of this system, which 
is planned to be used in distance education contexts, the 
student comments are automatically E-mailed to the 
author, or to the course organizer/uitor, wherever he or 
^e may be. The tutor would then reply directly on E- 
mail to the student, thus developing a discussion on the 
question that was raised. The present implementation 
only allows for a discussion between individual students 
and the tutor-at-a-distance, but by a simple 
modification, this discussion can be opened up, if 
desired, to all suidents. 



It is too early to be able to report any conclusive 
research data. However, initial observations indicate that 
the Structural Communication methodology is very 
acceptable to students and can lead to a depth of 
discussion which rivals the best group discussions that 
can be achieved in a classroom-based case study session. 
Of particular interest is the Hnding that students, or 
student groups, who normally participate very little in 
classroom discussion, for example certain groups of 
foreign students inhibited by their poor English or by 
cultural factors, participate just as actively in this 
methodology as any other student Another observation 
of note is that the total number of comments, the depth 
of these comments, the variety of these comments is 
several times greater than wculd normally be achieved 
in the time that could practically be given to classroom 
discussion of a case study of this nature. 

Rnally, we are investigating just how time consuming 
and diiDTicuU the task of the uitor becomes in this sort of 
system^ given that every student can, in theory, 
maintain an individual discussion for as long as he 
she wishes with the tutor. The tutor ought to be in a 
situation where he or she can respond adequately to 
every student and not leave them "hanging in the air** as 
it were. In one implementation of the mainframe-based 
system that ran for several weeks recently, one tutor 
was handling ov«r 100 students working on 
management^ information system case suidies. The 
particulartutortnvcdved habitually uses electronic mail 
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for communication with her students and habitually 
deals with somewhere in the region of 300 personal 
messages per week. The implementation of the case 
studies did not reduce the number of messages dealt 
with, indeed increased the total number somewhat. 
However, an analysis of tl:e messages showed that a 
large proportion of them were messages al out the 
methodology and generally favorable to the 
methodology. The subset of messages that were second 
level and third level discussions of the content of the 
case study were significant but did not form the 
majority of the total messages. We are therefore hopeful 
that we are in a position to develop a methodology 
which, in comparison with other email-based computer 
conferencing systems, actually reduces the load for the 
discussion moderator while at the same time increasing 
the amount of participant discussion and also the depth 
of that discussion. 
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TRAINING FUTURE EDUCATORS FOR 
INTEGRATED DISTANCE EDUCATION WORKSTATIONS 



FarhadSaba, San Diego State University 



Abstract 

This paper will describe how future instructional 
designers are trained at the Department of Educational 
Technology, San Diego State University to use 
integrated data*voice*video desktop workstations for 
distance education. Characteristics of converging 
technologies will be examined in light of recent 
technical changes. It will be demonstrated how students 
use HyperCard to create authoring templates for seven 
different distant instructional strategies and use 
Timbuktu for delivering instruction on integrated 
workstations. Also, it is described how students 
investigate and analyze selected topics related to current 
theories, research methods, evaluation strategies and 
current issues in distance education. 

Converging Technologies 

Marriage of computer, video and telecommunications 
technologies have provided a unique and unprecedented 
opportunity for reaching and serving learners, regardless 
of time or geographical considerations. As computers 
proliferate in the workplace, at home and in schools, 
trained educates who are familiar with theoretical and 
practical aspects of converging technologies will be in 
demand for designing instructfen for intej^nated systems. 
Trained professionals will be needed to: 

• Design programs based on current distajice 
education theory, 

• Develop programs for converging 
technologies, 

• Use converging technologies to deliver 
instruction, 

• Provide support services to distant learners, and 

• Conduct evaluation and research studie: on 
distance education concepts, models and 
programs. 



In the Department of Educational Technology at San 
Diego State University, students are learning how to 
use an integrated desktop workstation, based on a 
Macintosh compuler, to design, produce and present 
telelessons. In EDTEC 700, Seminar In Educational 
Technology: Distance Education, (Seidman, 1989) 
students use HyperCard to create authoring templates 
and Timbuktu to present telelessons. Course objectives 
include: ^ 

1. Name various parts of a typical integrated 
desktop workstation and describe the function 
of each part 

2. Develop a script to reflect instructional 
strategy, contenC medium, instructor's role; 
learner activities; and learner queries in a 
telelesson. 

3. Select appropriate micro design variables 
(fonts, graphics, sounds, visuals) for a 
telelesson. 

4. Use a desktop workstation to log on and 
establish basic voice , text and video 
telecommunications with another desktop 
workstation. 

5. Develop a HyperCard stack and share the 
stack with another user on a desktop 
workstation. 

6. Use a desktop woHcstation to send and receive 
instructional information. 

7. Develop and present a telelesson via Timbuktu 
and HyperCard. 

Hardware Configuration 

With the use of a newly developed integrated 
workstation, students learn how to communicate via 
data, voice, text, graphics and video with their learners 
in a laboratory environment (Saba and Twitchell, 1988). 




Figure 1* A prototype integrated desktop woricstation, cunently in use by students m the Department of 
Educational Technology, San Diego State University. 




435 



444 



Saba 



The teacher and the learner can talk to each other; work 
on the same computer file (screcnsharc), exchange data; 
and with the expansion of broadband telephony, 
eventually, sec each other. 

Figure 1 shows a prototype integrated workstation, 
currently in use by students in the Department of 
Educational Technology at San Diego State University. 
The instructor workstation is consisted of a telephone 
with a speaker to provide hands-free operation, a black 
and white video camera, a video monitor (not shown in 
Figure 1, as it is expected that the video and the 
computer screens converge in a near future) and a 
Macintosh SE/30. The instructor workstation can be 
augmented with a video playback unit to show video 
clips to the learner. The leaner workstation is identical 
to the instructor workstation, except that it does not 
include a video playback unit. The instructor and the 
learner workstations are connected by hard wire. This, 
however, dose not affect teaching of instructional design 
principles for integrated systems, as the wiring is 
transparent to students and future integrated 
telecommunications systems are effectively simulated 
with existing technology. Thus, with a relatively 
inexpensive prototype system students are trained to use 
a technology which is bound to become popular in the 
1990's. 

Software Configuration 

Those students who have completed at least one course 
in three areas of instructional design, educational 
broadcasting, and hypermedia production can enter the 
course. Through a series of demonstrations and 
laboratory exercises, students receive hands-on 
instruction on how to use the voice, video and 
computing features of the desktop workstation; and 
establish and maintain communications with at least 
one remote desktop workstation. As students are already 
familiar with the use of HyperCard (Allen, Dodge and 
Saba, 1990) and have learned how to produce and use 
video clips in prior courses, they can quickly learn how 
to use the desktop workstation for establishing bi- 
directional and synchronous communications with their 
learner working at a remote desktop workstation. The 
only new feature to them is Timbuktu, a 
communicaiions software that enables the insuuctor to 
share his/her hard disk drive with the learner for a 
variety of different computing purposes, including 
screen-sharing and file transfer. Timbuktu is a user 
friendly software and its use becomes transpare.it to 
both the instructor and the learner after a few minutes 
from initial exposure.. 

Once students have become familiar with basic 
communications and computing features of the 
workstation, they form groups of three to five. They are 
introduced to seven instructional strategies to produce a 
HyperCard template for designing, developing and 
presenting telelessons; or for managing on-line 
instructional systems. These strategies are; 
• Expository instruction. 



Coachin'^, 

• Tutoring. 

• Counseling, 
Problem solving. 
Cooperative learning. 

Building an electronic community, and 

• Managing the instructional process. 

Within the confines of these design strategies, students 
are encouraged to use their creative talents to include 
features in the template that would help them to strike 
an appropriate balance between dialog and structure, 
(Moore, 1983; Saba, 1988) as well as to maintain good 
communications with learners for administrative and 
counseling purposes. Figure 2 shows a card of a sample 
stack created for authoring of an expository telelesson. 
Figure 3 shows the menu of a HyperCard stack created 
for the management of an on-line instructional system. 

Authoring templates are presented in class by each 
group. Students receive feedback from their peers and 
the instructor on strengths and weaknesses of each 
approach. Once the templates are refmed, each student 
selects a content area, or an administrative/counseling 
procedure for designing, developing and presenting a 
telelesson; or an administrative procedure. Final projects 
are delivered "live" and on-line as students teach their 
learners a combination of facts, concept, procedures, 
rules, principles or theories related to the selected 
content area. As students are free to experiment with a 
wide variety of content areas, final projects have ranged 
from teachmg techniques of shot composition with a 
video camera, to administering acupuncture. Final 
projects are evaluated by the instructor as each student 
delivers his/her program. The evaluation form contains 
criteria related to macro instructional design, micro 
design (text, graphics, audio, video), establishing and 
maintaining technical communications, balancing 
dialog and suucture and evaluating the learner. 

Distance Education Theory 

In addition to preparing s.jdents to use integrated 
telecommunications systems, the course is designed to 
develop studjnts' analytical skills for deveL^.^tg a 
comprehensive understanding of current theories, 
research methods, evaluation strategies and current 
issues in distance education. During class lectures 
several issues related to the impact of "information 
society" on learning, training and schooling; 
characteristics of converging technologies; conceptual 
significance of converging technologies for interactive, 
synchronous learning; and current trends in distance 
education theory are inuoduced. Based on class lectures, 
a set of reading materials (Moore and Clark, 1989) and 
other publications each student selects an area of interest 
for investigation. Student^' receive individual 
consultation by instructor as the/ proceed in collecti z 
information about their topics. Towards the end of the 
semester, each student makes an oral presentation to the 
class, based on a written report 
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7^ 





How to Use 
Counseling/ Ad vising 

Registration & Fees 
Courses 

Access to Programs 
Mailbox 



Figure 2. A sample card of an authoring template in HyperCard for exposiiory distance instruction. 



Presentations are followed by class discussion. In the 
past two years, students have studied a variety of topics 
ranging from concepts of distance education, distance 
education organizations and distance education in the 
international arena to future technologies and 
telecomputing. 

Response to the Course 

EDTEC 700» has been offered on an experimental basis 
for the past tow years. Each year, students have received 
it with enthusiasm, dedication and hard woik. They feel 
they are contributing to the future development of the 
field of distance education by developing and presenting 
demonstration projects on a new technology. Because of 
its success, the Department is currently considering the 
adoption of the course on a regular schedule. 
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Template Instructions 



This instructional template is designed for 
instructors to present on-line lessons to be shared 
with 1-3 students. The instructional template 
features include the capability to: 



Share text or graphic screen displays. 

Add new card(s) to the lesson. 

Print any card on the display. 

Open another staclc from the template. 

Display an instructor's lesson outline. 

Type comments on an instructor's note pad. 



For additional information on the template 
features press the individual button on the right 
side of the instructional template. 



( Click to Continue j 




Open Staclc 

N ote Pa d 
Help 




Figure 3. Main menu of a HyperCard .-itack created for management of a prototype on-line instructional system. 
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